class: center, middle, inverse, title-slide # Make-like declarative workflows with <img src="images/R_logo_45.png" /> ## Exercises ### Kirill Müller ### Zurich, 2018-03-05 --- # The *cooking* package https://krlmlr.github.io/cooking https://krlmlr.github.io/cooking/reference/cooking-package.html ```r buy(what, from = supermarket()) supermarket() peel(what, ...) chop(what, ...) fry(what, ...) cook(what, ...) combine(..., name = "ragout") make_cooking(processed) ``` --- # Recipe snippets ```r library(cooking) chopped_meat <- chop("raw_meat.csv") chopped_meat ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> chopped meat, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> raw meat</div></code> ```r fry(chopped_meat, with = "vegetables") ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> fried meat, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> chopped meat, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> raw meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> vegetables</div></code> ```r peel(buy("carrots")) ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> peeled carrots, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> raw carrots, <span style="font-style:italic">bought at</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> supermarket</div></code> --- # New methods ```r dice <- make_cooking("diced", "cut from") dice("tomatoes") ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> diced tomatoes, <span style="font-style:italic">cut from</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> tomatoes</div></code> --- background-image: url(dots/detailed.png) background-size: contain background-position: 0% 90% # Exercises https://krlmlr.shinyapps.io/cooking-tutorial/ 1. Use `buy()` to buy meat 2. Chop this meat 3. Fry the chopped meat with oil 4. Combine the fried meat with vegetables 5. Add a salad --- # The *drake* package https://ropensci.github.io/drake ```r # Create plan `drake_plan()` # Execute plan `make()` # Read single target `readd()` # From plan to configuration `drake_config()` # Visualize `vis_drake_graph()` # What do we need to run? `outdated()` # What if a run fails? `diagnose()` ``` --- background-image: url(dots/detailed.png) background-size: contain background-position: 0% 90% ```r library(drake) *plan <- drake_plan( meat = buy("meat"), chopped_meat = chop(meat), fried_meat = fry(chopped_meat, with = "oil"), ragout = combine(fried_meat, with = "vegetables") ) plan ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#8a8a8a"># A tibble: 4 x 2</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> target command </div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="font-style:italic"><span style="color:#8a8a8a"><chr></span></span> <span style="font-style:italic"><span style="color:#8a8a8a"><chr></span></span> </div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#b2b2b2">1</span> meat <span style="color:#8a8a8a">"</span>buy(\"meat\")<span style="color:#8a8a8a">"</span> </div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#b2b2b2">2</span> chopped_meat chop(meat) </div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#b2b2b2">3</span> fried_meat <span style="color:#8a8a8a">"</span>fry(chopped_meat, with = \"oil\")<span style="color:#8a8a8a">"</span> </div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#b2b2b2">4</span> ragout <span style="color:#8a8a8a">"</span>combine(fried_meat, with = \"vegetables\")<span style="color:#8a8a8a">"</span></div></code> --- background-image: url(dots/detailed.png) background-size: contain background-position: 0% 90% ```r *make(plan) ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">cache</span> .../.drake</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> Unloading targets from environment:</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> chopped_meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">connect</span> 4 imports: pre_chunk, dice, colourise_chunk, plan</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">connect</span> 4 targets: meat, chopped_meat, fried_meat, ragout</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">check</span> 4 items: buy, chop, combine, fry</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">check</span> 1 item: meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#00afff">target</span> meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">check</span> 1 item: chopped_meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#00afff">target</span> chopped_meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">check</span> 1 item: fried_meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#ff5fff">unload</span> 1 item: meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#00afff">target</span> fried_meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">check</span> 1 item: ragout</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#ff5fff">unload</span> 1 item: chopped_meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#00afff">target</span> ragout</div></code> --- background-image: url(dots/detailed.png) background-size: contain background-position: 0% 90% ```r *readd(ragout) ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> <span style="color:#afd7d7">cache</span> .../.drake</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> ragout, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> fried meat, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> chopped meat, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> raw meat, <span style="font-style:italic">bought at</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> supermarket</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> oil</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> vegetables</div></code> --- ```r *config <- drake_config(plan, verbose = FALSE) *vis_drake_graph(config) ``` <div id="htmlwidget-ec897849c5c50a6f7846" style="width:504px;height:504px;" class="visNetwork html-widget"></div> <script type="application/json" data-for="htmlwidget-ec897849c5c50a6f7846">{"x":{"nodes":{"id":["buy","chop","fry","combine","meat","chopped_meat","fried_meat","ragout"],"label":["buy\n0.001s","chop\n0.001s","fry\n0.002s","combine\n0.001s","meat\n0.084s","chopped_meat\n0.054s","fried_meat\n0.014s","ragout\n0.012s"],"status":["imported","imported","imported","imported","up to date","up to date","up to date","up to date"],"type":["function","function","function","function","object","object","object","object"],"level":[0,0,0,0,1,2,3,4],"font.size":[20,20,20,20,20,20,20,20],"color":["#1874CD","#1874CD","#1874CD","#1874CD","#228B22","#228B22","#228B22","#228B22"],"shape":["triangle","triangle","triangle","triangle","dot","dot","dot","dot"],"hover_label":["function (what, from = supermarket()) \n{\n new_food(paste(\"raw\", what), input = from, provenance = \"bought at\")\n}","function (what, ...) \n{\n what <- get_what(what)\n main <- get_main(what)\n new_food(paste(\"chopped\", main), input = list(what, ...), \n provenance = NULL)\n}","function (what, ...) \n{\n what <- get_what(what)\n main <- get_main(what)\n new_food(paste(\"fried\", main), input = list(what, ...), provenance = NULL)\n}","function (..., name = \"ragout\") \n{\n new_food(name, list(...))\n}","buy(\"meat\")","chop(meat)","fry(chopped_meat, with = \"oil\")","combine(fried_meat, with = \"vegetables\")"],"x":[-1,-0.333333333333333,0.333333333333333,1,-1,-0.666666666666667,-0.333333333333333,0],"y":[-1,-1,-1,-1,-0.5,0,0.5,1]},"edges":{"from":["buy","chop","meat","chopped_meat","fry","combine","fried_meat"],"to":["meat","chopped_meat","chopped_meat","fried_meat","fried_meat","ragout","ragout"],"arrows":["to","to","to","to","to","to","to"]},"nodesToDataframe":true,"edgesToDataframe":true,"options":{"width":"100%","height":"100%","nodes":{"shape":"dot","physics":false},"manipulation":{"enabled":false},"layout":{"hierarchical":{"enabled":true,"direction":"LR"}},"edges":{"smooth":false},"physics":{"stabilization":false},"interaction":{"navigationButtons":true,"hover":true}},"groups":null,"width":null,"height":null,"idselection":{"enabled":false},"byselection":{"enabled":false},"main":{"text":"Workflow graph","style":"font-family:Georgia, Times New Roman, Times, serif;font-weight:bold;font-size:20px;text-align:center;"},"submain":null,"footer":null,"background":"rgba(0, 0, 0, 0)","legend":{"width":0.2,"useGroups":false,"position":"left","ncol":1,"stepX":100,"stepY":100,"zoom":true,"nodes":{"label":["Up to date","Outdated","In progress","Failed","Imported","Missing","Object","Function","File"],"color":["#228B22","#000000","#FF7221","#AA0000","#1874CD","#9A32CD","#888888","#888888","#888888"],"shape":["dot","dot","dot","dot","dot","dot","dot","triangle","square"],"font.color":["black","black","black","black","black","black","black","black","black"],"font.size":[20,20,20,20,20,20,20,20,20],"id":[1,2,3,4,5,6,7,8,9]},"nodesToDataframe":true},"igraphlayout":{"type":"square"},"tooltipStay":300,"tooltipStyle":"position: fixed;visibility:hidden;padding: 5px;white-space: nowrap;font-family: verdana;font-size:14px;font-color:#000000;background-color: #f5f4ed;-moz-border-radius: 3px;-webkit-border-radius: 3px;border-radius: 3px;border: 1px solid #808074;box-shadow: 3px 3px 10px rgba(0, 0, 0, 0.2);","events":{"hoverNode":"function(e){\n var label_info = this.body.data.nodes.get({\n fields: ['label', 'hover_label'],\n filter: function (item) {\n return item.id === e.node\n },\n returnType :'Array'\n });\n this.body.data.nodes.update({\n id: e.node,\n label : label_info[0].hover_label,\n hover_label : label_info[0].label\n });\n }","blurNode":"function(e){\n var label_info = this.body.data.nodes.get({\n fields: ['label', 'hover_label'],\n filter: function (item) {\n return item.id === e.node\n },\n returnType :'Array'\n });\n this.body.data.nodes.update({\n id: e.node,\n label : label_info[0].hover_label,\n hover_label : label_info[0].label\n });\n }"}},"evals":["events.hoverNode","events.blurNode"],"jsHooks":[]}</script> --- ```r config <- drake_config(plan, verbose = FALSE) *outdated(config) ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> character(0)</div></code> --- background-image: url(dots/full.png) background-size: contain background-position: 0% 90% # Exercises https://krlmlr.shinyapps.io/cooking-drake-tutorial/ 1. Plan to buy meat with `buy()` 2. Plan to chop and fry the meat 3. Fry with garlic instead 4. Buy at the butcher's instead 5. Prepare vegetables 6. Combine the ragout 7. How to **not** combine the ragout --- # *drake* and files ```r library(drake) plan <- drake_plan( * meat = buy(readLines(file_in("raw-meat.txt"))), chopped_meat = chop(meat), fried_meat = fry(chopped_meat, with = "oil"), ragout = combine(fried_meat, with = "vegetables"), * saveRDS(ragout, file_out("ragout.rds")), rmarkdown::render( * knitr_in("ragout.Rmd"), output_file = file_out("ragout.md") ) ) ``` --- # Exercises https://github.com/krlmlr/drake-sib-zurich/tree/master/cooking-with-files 1. Save `make.R` as `make-solution.R`, do not edit `make.R`. 2. Use files to describe the meat and store the ragout. 3. Run the script. 4. Edit the `raw-meat.txt` file. 5. Run `make()`. Do the results align with your expectations? 6. Create a new text file and use it instead of `raw-meat.txt`. Run `make()` again. 7. Open the file `cooking-with-files.Rmd`, knit it. How does the code in the `.Rmd` document access the targets we just built? 8. Extend the plan to knit the document via `rmarkdown::render()`. Remember to use `knitr_in()` for the input file. 9. Verify that the `.Rmd` is processed if you change inputs or commands. --- --- # The pipe ```r "carrots" %>% buy() %>% peel() %>% chop() %>% fry(with = c("oil", "garlic", "chopped meat")) %>% combine("mashed potatoes") ``` <code class="remark-code r"><div class="remark-code-line"><span style="color:#8a8a8a">#></span> ragout, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> fried carrots, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> chopped carrots, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> peeled carrots, <span style="font-style:italic">made of</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> raw carrots, <span style="font-style:italic">bought at</span></div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> supermarket</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> oil </div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> garlic </div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> chopped meat</div> <div class="remark-code-line"><span style="color:#8a8a8a">#></span> mashed potatoes</div></code>