public-collect/books-futures-explained
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

several improvements, see #2 for more details

Browse Source
This commit is contained in:
Carl Fredrik Samson
2020-04-10 20:26:41 +02:00
parent 08cda06ade
commit 32bedb934c
22 changed files with 2236 additions and 2356 deletions
Download Patch File Download Diff File Expand all files Collapse all files

311
book/3_generators_async_await.html
View File

@@ -1,5 +1,5 @@
<!DOCTYPE HTML>
<html lang="en" class="sidebar-visible no-js light">
<html lang="en" class="sidebar-visible no-js">
<head>
<!-- Book generated using mdBook -->
<meta charset="UTF-8">
@@ -32,11 +32,11 @@
</head>
<body>
<body class="light">
<!-- Provide site root to javascript -->
<script type="text/javascript">
var path_to_root = "";
var default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? "light" : "light";
var default_theme = "light";
</script>
<!-- Work around some values being stored in localStorage wrapped in quotes -->
@@ -60,11 +60,8 @@
var theme;
try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
if (theme === null || theme === undefined) { theme = default_theme; }
var html = document.querySelector('html');
html.classList.remove('no-js')
html.classList.remove('light')
html.classList.add(theme);
html.classList.add('js');
document.body.className = theme;
document.querySelector('html').className = theme + ' js';
</script>
<!-- Hide / unhide sidebar before it is displayed -->
@@ -80,8 +77,8 @@
</script>
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
<div id="sidebar-scrollbox" class="sidebar-scrollbox">
<ol class="chapter"><li class="expanded affix "><a href="introduction.html">Introduction</a></li><li class="expanded "><a href="0_background_information.html"><strong aria-hidden="true">1.</strong> Background information</a></li><li class="expanded "><a href="1_futures_in_rust.html"><strong aria-hidden="true">2.</strong> Futures in Rust</a></li><li class="expanded "><a href="2_waker_context.html"><strong aria-hidden="true">3.</strong> Waker and Context</a></li><li class="expanded "><a href="3_generators_async_await.html" class="active"><strong aria-hidden="true">4.</strong> Generators and async/await</a></li><li class="expanded "><a href="4_pin.html"><strong aria-hidden="true">5.</strong> Pin</a></li><li class="expanded "><a href="6_future_example.html"><strong aria-hidden="true">6.</strong> Implementing Futures</a></li><li class="expanded "><a href="8_finished_example.html"><strong aria-hidden="true">7.</strong> Finished example (editable)</a></li><li class="expanded affix "><a href="conclusion.html">Conclusion and exercises</a></li></ol>
<div class="sidebar-scrollbox">
<ol class="chapter"><li class="affix"><a href="introduction.html">Introduction</a></li><li><a href="0_background_information.html"><strong aria-hidden="true">1.</strong> Background information</a></li><li><a href="1_futures_in_rust.html"><strong aria-hidden="true">2.</strong> Futures in Rust</a></li><li><a href="2_waker_context.html"><strong aria-hidden="true">3.</strong> Waker and Context</a></li><li><a href="3_generators_async_await.html" class="active"><strong aria-hidden="true">4.</strong> Generators and async/await</a></li><li><a href="4_pin.html"><strong aria-hidden="true">5.</strong> Pin</a></li><li><a href="6_future_example.html"><strong aria-hidden="true">6.</strong> Implementing Futures</a></li><li><a href="8_finished_example.html"><strong aria-hidden="true">7.</strong> Finished example (editable)</a></li><li class="affix"><a href="conclusion.html">Conclusion and exercises</a></li></ol>
</div>
<div id="sidebar-resize-handle" class="sidebar-resize-handle"></div>
</nav>
@@ -351,7 +348,7 @@ in depth explanation see <a href="https://tmandry.gitlab.io/blog/posts/optimizin
};
</code></pre>
<p>We'll be hand-coding some versions of a state-machines representing a state
machine for the generator defined aboce.</p>
machine for the generator defined above.</p>
<p>We step through each step &quot;manually&quot; in every example, so it looks pretty
unfamiliar. We could add some syntactic sugar like implementing the <code>Iterator</code>
trait for our generators which would let us do this:</p>
@@ -363,19 +360,19 @@ trait for our generators which would let us do this:</p>
Just keep this in the back of your head as we move forward.</p>
<p>Now what does our rewritten state machine look like with this example?</p>
<pre><pre class="playpen"><code class="language-rust compile_fail">
<span class="boring">#![allow(unused_variables)]
</span><span class="boring">fn main() {
</span><span class="boring">enum GeneratorState&lt;Y, R&gt; {
</span><span class="boring"> Yielded(Y),
</span><span class="boring"> Complete(R),
</span><span class="boring">}
</span><span class="boring">
</span><span class="boring">trait Generator {
</span><span class="boring"> type Yield;
</span><span class="boring"> type Return;
</span><span class="boring"> fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt;;
</span><span class="boring">}
</span>
# #![allow(unused_variables)]
#fn main() {
# enum GeneratorState&lt;Y, R&gt; {
# Yielded(Y),
# Complete(R),
# }
#
# trait Generator {
# type Yield;
# type Return;
# fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt;;
# }
enum GeneratorA {
Enter,
Yield1 {
@@ -385,12 +382,12 @@ enum GeneratorA {
Exit,
}
<span class="boring">impl GeneratorA {
</span><span class="boring"> fn start() -&gt; Self {
</span><span class="boring"> GeneratorA::Enter
</span><span class="boring"> }
</span><span class="boring">}
</span>
# impl GeneratorA {
# fn start() -&gt; Self {
# GeneratorA::Enter
# }
# }
impl Generator for GeneratorA {
type Yield = usize;
type Return = ();
@@ -415,8 +412,7 @@ impl Generator for GeneratorA {
}
}
}
<span class="boring">}
</span></code></pre></pre>
#}</code></pre></pre>
<p>If you try to compile this you'll get an error (just try it yourself by pressing play).</p>
<p>What is the lifetime of <code>&amp;String</code>. It's not the same as the lifetime of <code>Self</code>. It's not <code>static</code>.
Turns out that it's not possible for us in Rusts syntax to describe this lifetime, which means, that
@@ -427,9 +423,9 @@ see we end up in a <em>self referential struct</em>. A struct which holds refere
into itself.</p>
<p>As you'll notice, this compiles just fine!</p>
<pre><pre class="playpen"><code class="language-rust">
<span class="boring">#![allow(unused_variables)]
</span><span class="boring">fn main() {
</span>enum GeneratorState&lt;Y, R&gt; {
# #![allow(unused_variables)]
#fn main() {
enum GeneratorState&lt;Y, R&gt; {
Yielded(Y),
Complete(R),
}
@@ -483,8 +479,7 @@ impl Generator for GeneratorA {
}
}
}
<span class="boring">}
</span></code></pre></pre>
#}</code></pre></pre>
<p>Remember that our example is the generator we crated which looked like this:</p>
<pre><code class="language-rust noplaypen ignore">let mut gen = move || {
let to_borrow = String::from(&quot;Hello&quot;);
@@ -511,66 +506,66 @@ does what we'd expect. But there is still one huge problem with this:</p>
()
};
}
<span class="boring">enum GeneratorState&lt;Y, R&gt; {
</span><span class="boring"> Yielded(Y),
</span><span class="boring"> Complete(R),
</span><span class="boring">}
</span><span class="boring">
</span><span class="boring">trait Generator {
</span><span class="boring"> type Yield;
</span><span class="boring"> type Return;
</span><span class="boring"> fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt;;
</span><span class="boring">}
</span><span class="boring">
</span><span class="boring">enum GeneratorA {
</span><span class="boring"> Enter,
</span><span class="boring"> Yield1 {
</span><span class="boring"> to_borrow: String,
</span><span class="boring"> borrowed: *const String,
</span><span class="boring"> },
</span><span class="boring"> Exit,
</span><span class="boring">}
</span><span class="boring">
</span><span class="boring">impl GeneratorA {
</span><span class="boring"> fn start() -&gt; Self {
</span><span class="boring"> GeneratorA::Enter
</span><span class="boring"> }
</span><span class="boring">}
</span><span class="boring">impl Generator for GeneratorA {
</span><span class="boring"> type Yield = usize;
</span><span class="boring"> type Return = ();
</span><span class="boring"> fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt; {
</span><span class="boring"> match self {
</span><span class="boring"> GeneratorA::Enter =&gt; {
</span><span class="boring"> let to_borrow = String::from(&quot;Hello&quot;);
</span><span class="boring"> let borrowed = &amp;to_borrow;
</span><span class="boring"> let res = borrowed.len();
</span><span class="boring"> *self = GeneratorA::Yield1 {to_borrow, borrowed: std::ptr::null()};
</span><span class="boring">
</span><span class="boring"> // We set the self-reference here
</span><span class="boring"> if let GeneratorA::Yield1 {to_borrow, borrowed} = self {
</span><span class="boring"> *borrowed = to_borrow;
</span><span class="boring"> }
</span><span class="boring">
</span><span class="boring"> GeneratorState::Yielded(res)
</span><span class="boring"> }
</span><span class="boring">
</span><span class="boring"> GeneratorA::Yield1 {borrowed, ..} =&gt; {
</span><span class="boring"> let borrowed: &amp;String = unsafe {&amp;**borrowed};
</span><span class="boring"> println!(&quot;{} world&quot;, borrowed);
</span><span class="boring"> *self = GeneratorA::Exit;
</span><span class="boring"> GeneratorState::Complete(())
</span><span class="boring"> }
</span><span class="boring"> GeneratorA::Exit =&gt; panic!(&quot;Can't advance an exited generator!&quot;),
</span><span class="boring"> }
</span><span class="boring"> }
</span><span class="boring">}
</span></code></pre></pre>
# enum GeneratorState&lt;Y, R&gt; {
# Yielded(Y),
# Complete(R),
# }
#
# trait Generator {
# type Yield;
# type Return;
# fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt;;
# }
#
# enum GeneratorA {
# Enter,
# Yield1 {
# to_borrow: String,
# borrowed: *const String,
# },
# Exit,
# }
#
# impl GeneratorA {
# fn start() -&gt; Self {
# GeneratorA::Enter
# }
# }
# impl Generator for GeneratorA {
# type Yield = usize;
# type Return = ();
# fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt; {
# match self {
# GeneratorA::Enter =&gt; {
# let to_borrow = String::from(&quot;Hello&quot;);
# let borrowed = &amp;to_borrow;
# let res = borrowed.len();
# *self = GeneratorA::Yield1 {to_borrow, borrowed: std::ptr::null()};
#
# // We set the self-reference here
# if let GeneratorA::Yield1 {to_borrow, borrowed} = self {
# *borrowed = to_borrow;
# }
#
# GeneratorState::Yielded(res)
# }
#
# GeneratorA::Yield1 {borrowed, ..} =&gt; {
# let borrowed: &amp;String = unsafe {&amp;**borrowed};
# println!(&quot;{} world&quot;, borrowed);
# *self = GeneratorA::Exit;
# GeneratorState::Complete(())
# }
# GeneratorA::Exit =&gt; panic!(&quot;Can't advance an exited generator!&quot;),
# }
# }
# }
</code></pre></pre>
<p>The problem is that in safe Rust we can still do this:</p>
<p><em>Run the code and compare the results. Do you see the problem?</em></p>
<pre><pre class="playpen"><code class="language-rust should_panic"><span class="boring">#![feature(never_type)] // Force nightly compiler to be used in playground
</span><span class="boring">// by betting on it's true that this type is named after it's stabilization date...
</span>pub fn main() {
<pre><pre class="playpen"><code class="language-rust should_panic"># #![feature(never_type)] // Force nightly compiler to be used in playground
# // by betting on it's true that this type is named after it's stabilization date...
pub fn main() {
let mut gen = GeneratorA::start();
let mut gen2 = GeneratorA::start();
@@ -589,61 +584,61 @@ does what we'd expect. But there is still one huge problem with this:</p>
()
};
}
<span class="boring">enum GeneratorState&lt;Y, R&gt; {
</span><span class="boring"> Yielded(Y),
</span><span class="boring"> Complete(R),
</span><span class="boring">}
</span><span class="boring">
</span><span class="boring">trait Generator {
</span><span class="boring"> type Yield;
</span><span class="boring"> type Return;
</span><span class="boring"> fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt;;
</span><span class="boring">}
</span><span class="boring">
</span><span class="boring">enum GeneratorA {
</span><span class="boring"> Enter,
</span><span class="boring"> Yield1 {
</span><span class="boring"> to_borrow: String,
</span><span class="boring"> borrowed: *const String,
</span><span class="boring"> },
</span><span class="boring"> Exit,
</span><span class="boring">}
</span><span class="boring">
</span><span class="boring">impl GeneratorA {
</span><span class="boring"> fn start() -&gt; Self {
</span><span class="boring"> GeneratorA::Enter
</span><span class="boring"> }
</span><span class="boring">}
</span><span class="boring">impl Generator for GeneratorA {
</span><span class="boring"> type Yield = usize;
</span><span class="boring"> type Return = ();
</span><span class="boring"> fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt; {
</span><span class="boring"> match self {
</span><span class="boring"> GeneratorA::Enter =&gt; {
</span><span class="boring"> let to_borrow = String::from(&quot;Hello&quot;);
</span><span class="boring"> let borrowed = &amp;to_borrow;
</span><span class="boring"> let res = borrowed.len();
</span><span class="boring"> *self = GeneratorA::Yield1 {to_borrow, borrowed: std::ptr::null()};
</span><span class="boring">
</span><span class="boring"> // We set the self-reference here
</span><span class="boring"> if let GeneratorA::Yield1 {to_borrow, borrowed} = self {
</span><span class="boring"> *borrowed = to_borrow;
</span><span class="boring"> }
</span><span class="boring">
</span><span class="boring"> GeneratorState::Yielded(res)
</span><span class="boring"> }
</span><span class="boring">
</span><span class="boring"> GeneratorA::Yield1 {borrowed, ..} =&gt; {
</span><span class="boring"> let borrowed: &amp;String = unsafe {&amp;**borrowed};
</span><span class="boring"> println!(&quot;{} world&quot;, borrowed);
</span><span class="boring"> *self = GeneratorA::Exit;
</span><span class="boring"> GeneratorState::Complete(())
</span><span class="boring"> }
</span><span class="boring"> GeneratorA::Exit =&gt; panic!(&quot;Can't advance an exited generator!&quot;),
</span><span class="boring"> }
</span><span class="boring"> }
</span><span class="boring">}
</span></code></pre></pre>
# enum GeneratorState&lt;Y, R&gt; {
# Yielded(Y),
# Complete(R),
# }
#
# trait Generator {
# type Yield;
# type Return;
# fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt;;
# }
#
# enum GeneratorA {
# Enter,
# Yield1 {
# to_borrow: String,
# borrowed: *const String,
# },
# Exit,
# }
#
# impl GeneratorA {
# fn start() -&gt; Self {
# GeneratorA::Enter
# }
# }
# impl Generator for GeneratorA {
# type Yield = usize;
# type Return = ();
# fn resume(&amp;mut self) -&gt; GeneratorState&lt;Self::Yield, Self::Return&gt; {
# match self {
# GeneratorA::Enter =&gt; {
# let to_borrow = String::from(&quot;Hello&quot;);
# let borrowed = &amp;to_borrow;
# let res = borrowed.len();
# *self = GeneratorA::Yield1 {to_borrow, borrowed: std::ptr::null()};
#
# // We set the self-reference here
# if let GeneratorA::Yield1 {to_borrow, borrowed} = self {
# *borrowed = to_borrow;
# }
#
# GeneratorState::Yielded(res)
# }
#
# GeneratorA::Yield1 {borrowed, ..} =&gt; {
# let borrowed: &amp;String = unsafe {&amp;**borrowed};
# println!(&quot;{} world&quot;, borrowed);
# *self = GeneratorA::Exit;
# GeneratorState::Complete(())
# }
# GeneratorA::Exit =&gt; panic!(&quot;Can't advance an exited generator!&quot;),
# }
# }
# }
</code></pre></pre>
<p>Wait? What happened to &quot;Hello&quot;? And why did our code segfault?</p>
<p>Turns out that while the example above compiles just fine, we expose consumers
of this this API to both possible undefined behavior and other memory errors
@@ -663,7 +658,7 @@ by looking at how generators and the async keyword is related.</p>
<h2><a class="header" href="#async-and-generators" id="async-and-generators">Async and generators</a></h2>
<p>Futures in Rust are implemented as state machines much the same way Generators
are state machines.</p>
<p>You might have noticed the similarites in the syntax used in async blocks and
<p>You might have noticed the similarities in the syntax used in async blocks and
the syntax used in generators:</p>
<pre><code class="language-rust ignore">let mut gen = move || {
let to_borrow = String::from(&quot;Hello&quot;);
@@ -688,7 +683,7 @@ a Future works and the way a Generator work internally is similar.</p>
returning <code>Yielded</code> or <code>Complete</code> it returns <code>Pending</code> or <code>Ready</code>. Each <code>await</code>
point in a future is like a <code>yield</code> point in a generator.</p>
<p>Do you see how they're connected now?</p>
<p>Thats why kowing how generators work and the challanges they pose also teaches
<p>Thats why knowing how generators work and the challenges they pose also teaches
you how futures work and the challenges we need to tackle when working with them.</p>
<p>The same goes for the challenges of borrowing across yield/await points.</p>
<h2><a class="header" href="#bonus-section---self-referential-generators-in-rust-today" id="bonus-section---self-referential-generators-in-rust-today">Bonus section - self referential generators in Rust today</a></h2>
@@ -810,18 +805,6 @@ pub fn main() {
<script type="text/javascript">
window.playpen_line_numbers = true;
</script>
<script type="text/javascript">
window.playpen_copyable = true;
</script>
<script src="ace.js" type="text/javascript" charset="utf-8"></script>
<script src="editor.js" type="text/javascript" charset="utf-8"></script>
<script src="mode-rust.js" type="text/javascript" charset="utf-8"></script>