Add MemoizedWebCrawler tutorial #198
Conversation
| * (2) Then it defines a rule for any target that ends with `.tar` extension | ||
| * In the body of the rule, (3) it reads content of a `txt` file with the same prefix and (4) dynamically declares dependency via `need contents` | ||
| * This may trigger building artifacts that match other rules | ||
| * (5) Once the dependencies are met, it spawns `tar -cf` passing all the paths it as CLI arg |
|
|
||
| ## Background | ||
|
|
||
| The core reason why llbuild2fx is powerful is its reliance on content-addressable storage. Think of a tree where each node has a checksum. For each labeled node, the checksum is computed by putting together the checksum of the label, and aggregating the checksum of the children. |
There was a problem hiding this comment.
I think we can make this paragraph better:
In particular, CASTrees (an abstraction very similar to Unix file tree) have a very elegant encoding on top of a content-addressable storage. This fits nicely with the kind of use-cases that a conventional distributed build system is expected to address.
| main = shakeArgs shakeOptions $ do | ||
| want ["result.tar"] -- (1) | ||
| "*.tar" %> \out -> do -- (2) | ||
| contents <- readFileLines $ out -<.> "txt" -- (3) |
There was a problem hiding this comment.
readFile' :: Partial => FilePath -> Action String
Read a file, after calling need. The argument file will be tracked as a dependency.
and
readFileLines :: Partial => FilePath -> Action [String]
A version of readFile' which also splits the result into lines. The argument file will be tracked as a dependency
From https://hackage.haskell.org/package/shake-0.19.8/docs/Development-Shake.html#v:readFile-39-
There is no tutorial out there to explain how to use
llbuild2fx. The current GameOfLife example uses the less popularllbuild2BuildSystemlayer instead ofllbuild2fx.I wrote up a little tutorial in
README.md. I'm intending to fill out the TBD sections prior to merging.