Skip to main content

R14 Debugging Dreamcatchers

There are two types of debugging tooling we require

  1. debugging Interblock chain complexes
  2. debugging attribution and governance operations on Dreamcatcher objects

We want to be able to:

  1. Replay any situation
  2. Remove the need for logging, since we can replay the app
  3. Make reproducible bug reporting trivial, as we can include chains with each report
  4. Roll back, fix a bug, roll forward with a patched state, and with patched software, and replay any actions that were received since then ?
  5. Be able to debug the engine itself using these very same tools, by making the engine itself run inside a blockchain

mcnasty — 02/28/2022 @rexmondo how mad would you be as a dev if I removed being able to receive replies in a reducer any other way but awaiting the promise returned from making the request ? rexmondo — 02/28/2022 I think non blocking async await calls happen to me like 1/50 times or so The clearest example I can think of is writing to a log, I don’t want to cede control flow to do that mcnasty — 02/28/2022 oh you are welcome to fire and forget rexmondo — 02/28/2022 Maybe I don’t grok then mcnasty — 02/28/2022 I want to not dispatch any @@REPLY actions to the reducer ever if you did not await the call, then you imply to me that you don't give AF about its response rexmondo — 02/28/2022 I’m assuming replies are mapped into the expected action for the continuation? mcnasty — 02/28/2022 but if you do await the call, then I will return the continuation action payload to you, or I will throw if it is rejected by the remote chain rexmondo — 02/28/2022 Yeah, that’s very sane mcnasty — 02/28/2022 ok good rexmondo — 02/28/2022 That’s how we built it before too mcnasty — 02/28/2022 HOWEVER if you fire and forget, and the remote side rejects, I will crash your reducer rexmondo — 02/28/2022 You get the continuation action and the call environment rexmondo — 02/28/2022 This is an error bubbling problem mcnasty — 02/28/2022 i figure thats how node does it ? unhandled promise rejection ? rexmondo — 02/28/2022 This is how real promises work If the remote can reject you have to provide a continuation context to catch errors I think that’s really sane too mcnasty — 02/28/2022 from the programmers point of view in the reducer, these are real promises rexmondo — 02/28/2022 Ya, it sounds like you got the model right mcnasty — 02/28/2022 the gotcha is that I"m rerunning the devs code each time new replies come in and if they don't give me the exact same requests in the exact same order, I burn them because I can't really help someone who makes their code run differently they may have missed the point of blockchain mcnasty — 02/28/2022 Ok I'll roll with that rexmondo — 02/28/2022 Lol Yeah, what you’re saying is also why it’s a bad idea to expose the runtime mechanics too much, especially when it comes to continuation handling The lower level you show them, the more likely they will hand you something unstructured mcnasty — 02/28/2022 yeah 100% rexmondo — 02/28/2022 And then in 40 years you will have windows mcnasty — 02/28/2022 the oppression is there to help them 🤣 rexmondo — 02/28/2022 Exactly Now, actually you probably do want to consider the case where I didn’t catch the error here but my caller wants to mcnasty — 02/28/2022 V2 🤷 rexmondo — 02/28/2022 It’s actually really convenient in a program to let the higher levels of the program aggregate errors so you can logically centralize the user facing messages from those errors mcnasty — 02/28/2022 continuation is a nightmare I'm sorry I think I miss what you mean maybe an example ? rexmondo — 02/28/2022 So if I’m going to create an api, I always put the error handler right where I pick the routes I want to call into, the fan out/fan in point That way there’s only one place to manage the communication back to the api caller, and I can keep a big list of error type->human readable message all in one spot If you were to catch the errors right where they happen, you probably aren’t in the right context to even know how this is going to get sent back to the api caller The way we did this before was to put a synthetic continuation that gets called only when an error hook is not supplied that finds the caller and replies to it with an error that we copy from the original error You could do the same thing in userland too, we just added it automatically if you didn’t mcnasty — 02/28/2022 hmmm........ I think that is handled by promise chaining, or by crashing if an unhandled exception occurs. I'll try double check this with you whenever our next call occurs rexmondo — 02/28/2022 Yeah as long as you can chain promises you can bubble errors by just providing your own bubble code mcnasty — 02/28/2022 I am currently passing the whole error itself so you get the genuine error being thrown, stack trace and all there isn't much of the JVM that I haven't serialized in some way by now 🤦 rexmondo — 02/28/2022 Hahaha The other thing I played with when I was trying it is adding a chain trace in addition to the stack trace As you hop back through the chains, adding them to the stack trace too mcnasty — 02/28/2022 hey that's a really good idea I'll put that in the backlog rexmondo — 02/28/2022 Yeah, it’s like getting the async frames in node, it’s way more helpful than I expected mcnasty — 02/28/2022 my dream is that along with those errors, we could give you a blockhash and you could fire up a tool that would recreate the exact situation that caused the error plus autogenerate some jest tests using these hashes as the start point, since hey its all as permanent as you want it 🤷 rexmondo — 02/28/2022 That sounds real good mcnasty — 02/28/2022 basically, fuck logs, especially in cloud environments replay the whole app instead honestly the size of the logs is usually so massive, and you're still not sure you captured everything you need rexmondo — 02/28/2022 I love love love the idea of all errors coming with everything you need to understand them And replay them mcnasty — 02/28/2022 and if its safe to do so, the general public should be able to replay them and fix them go to sleep, some error happens, wake up - fixed already they just own a small piece of your attribution I mean if you can repro the error by default, things should be much easier that's often most of the struggle anyways #yoco you only crash once mcnasty — 02/28/2022 That's actually crazy powerful now that I think about it in the context of onboarding noobs - their app throws an error, it auto generates an IPFS link to the blockhash, channelId, and action number, then we can replay it exactly in browser welcome to the permanent errors web 🤣 rexmondo — 02/28/2022 its also a super powerful toolkit for interacting with foreign chains you can trace all the way back to the foreign input mcnasty — 02/28/2022 yeah that's a different set of tooling that I dream about tho like each time you change something, replay all your foreign chain interactions and make sure you still give back the same results oh wait sorry - you're talking about when you first encounter an error rexmondo — 02/28/2022 no impedance mismatch across the known set of interactions! thats amazing mcnasty — 02/28/2022 yeah, so you can actually force semver increments rexmondo — 02/28/2022 i meant it as like if their input causes my error, i can adapt my model of their system rexmondo — 02/28/2022 derived semver! mcnasty — 02/28/2022 "hey we just replayed your code against everyone who ever called the last version, and some of the output was different so therefore your interface has changed - welcome to your new major version" mcnasty — 02/28/2022 👍 hopefully we can sell some of this, as semver verification as a service, or other such changeables rexmondo — 02/28/2022 i think its easily sellable dependency versioning is a nightmare mcnasty — 02/28/2022 so is repro but solving both problems together should be quite liberating repro is pretty much the heart of blockchain - we should be using it as a programming tool rexmondo — 02/28/2022 100% im just imagining a version bump process where the entire history of using the dependency is replayed using the new version and you can see the entire affected surface of your app mcnasty — 02/28/2022 oh wow that would be super helpful then you could even auto upgrade so long as nothing changed then it gets fun, since we need staked pools to guarantee nothing sneaky made it in there that's the exciting shit - politics, money, code all swirling in one big cauldron lol anyway I'm out for the day - catch you later 👊 rexmondo — 02/28/2022 Peace