Old Gods
One of my favorite quotes on computers comes from Joseph Campbell :
Computers are like Old Testament gods; lots of rules and no mercy.
Through the levity there is more than a bit of truth in the statement. Computer programs are dumb, pedantic and utterly unforgiving. Most of the major innovations in software engineering over the past few decades seem to come from attempting in some way to protect us from ourselves. Whether it be commonplace efforts like test driven development or linting to arcane methods such as Hoare Logic see an excellent tutorial here . Software engineering seems to be a mental self-defense art.
It is beyond my knowledge and it would seem beyond any moral computer scientist to propose a sure fire method to guarantee correct code. Even when it's too expensive to fail like in space flight, we've seen almost comical errors such as NASA's 1999 metric mess up. Thus it would seem like a sisyphean endeavor to try get things perfect. Is the best we can hope for a mostly stable application to be quickly patched and eventually discarded when the technical debt mounts too high? Silicon Valley's move fast and break things philosophy seems to agree with this assessment
Yet code is law on face value is a mantra that demands perfection and like the gods of the old testament inflicts brutal punishment for failure. My central argument is that this interpretation isn't accurate. Rather it's a misunderstanding at best and a politically motivated strawman at worst.
Allegality
When Ethereum started the allegality movement, the tagline was code that can't care. My interpretation of the concept is that caring can't be at a protocol level. It is the responsibility of the author and the users as well as their respective jurisdictions to sort these things out. The protocol- like Bitcoin before it- would be an automated goliath dispassionately parsing line after line of logic until something meaningful is output.
This notion seems to violate our human sense of morality. We can imagine a modern case of the inquiring murderer where a terrorist connects a weapon of mass destruction to a timer smart contract. Should we censor the contract to prevent the bomb from going off? Is the code is law paradigm really worth human life? Further discussion can be read in Shi's excellent essay The Ring of Gyges.
The most likely source for this debate stems from a conflation of human with computer ethics. Cryptocurrency protocols, autonomous vehicles, drones and other systems that are in part or completely controlled by code aren't governed by human morality. They are absent of it. This reality also makes them perfectly predictable, immune to corruption and coercion. There is no ego to protect. There is no bias in judgement.
We aren't used to interacting with these types of systems. Humans are creatures of exceptions, biases and relationships. We build dense networks of contacts to protect ourselves from unforeseen issues. We leverage political connections to gain advantages. The more sociopathic manipulate emotions or use extortion to get what they want . These behaviors- for better or worse- are expected and entirely human.
When an actor is introduced so alien to our expectations, we initially could enjoy the benefits, but many will immediately try to change its nature the moment that they are exposed to consequences. The DAO incident isn't some dramatic black swan event; it's the rule. Thus we are left at a strange impasse. Is it ok for humans to be governed by systems that will not care about us?
One Sided Notions
Returning to the beginning, perhaps the hardest component of this paradigm lies in our inability to fully express our intention and will to computers. We simply don't speak the same language and cannot get enough precision to guarantee unintended things won't happen. It seems unfair to punish someone for a misinterpretation. Yet this notion seems awfully one sided.
We can easily see the flaws in computer interpretation, but what if the human side is defective? For example, the recent Tesla autopilot crash attracted international press. Reporters and consumer advocacy groups were quick to call for a dramatic slowdown in the proliferation of autopilot technology citing safety concerns; however, empirical evidence shows clearly that even in Tesla's semi-flawed state, autopilot is dramatically safer than driving drunk. How many are killed every year from distracted or impaired driving?
This nuance is what I think is lost in the arguments against code is law. We like to argue from a particular to a general case, yet fail to understand the long term positive benefits of the general case. Bitcoin's existence has forced a dramatic evolution in payments and banking. It has forced the issue of reducing remittance costs and broader inclusion into the world financial system. It has done all of this without protocol level regulation or accepting human ethics. We've had to engineer humanity in above the protocol.
Similarly Ethereum under a code is law paradigm forces the issue of correctness at a much higher level. It forces engineers to explicitly code exceptions, paths to upgrades or defenses to mistakes into their smart contracts. It forces better practices and punishes bad ones. It seems to be an evolutionary incentive towards a better standard of engineering. Furthermore, it forces humans to think much more carefully about intent and outcomes.
The key point is that code of law doesn't say that you can't have intervention and arbitration. It doesn't say that you have to abandon local laws and regulations. It says that you have to program them into your smart contract. Furthermore, where it's difficult to model intent programmatically, one can still develop a domain specific language to capture it for humans to arbitrate (like a legal markup if one wills).
These are necessary steps to safely interact with an old testament code god. And, they actually bring a great deal of benefit. You end up with much better contracts, cleaner expression of intent, open source libraries that are reusable and emerging government regulations customized to account for the new technology. None of these things are necessary when you permit human intervention at the protocol level.
Nosce Te Ipsum
Thus, I conclude it takes courage to know a new land. We cannot truly appreciate the road ahead nor the events that will hurt us, but we can acknowledge that we are flawed creatures.
The existence of old testament protocols are monuments to our intellect. We have created something that allows us to transcend our very nature and interact with something completely different.
While these interactions have consequences, they make us better and force us to think differently. To me, it is a new epoch in humanity that we must embrace. It's the 21st century's nosce te ipsum moment. Let's not squander it.
Very thought provoking!
Someday we may be at the autonomous robot ethics point, but nowadays I'm thinking that we are simply talking about autopilot ethics.
Pilots use them maybe 90% of their flight time, but turn them off when things get tricky.
We might as well face those facts and enjoy the workload relief from the 90% of the time that they do work.
That said, having the code serve as "guard rails" to keep the developers safe from coercion and consumers safe from corruption is a fruitful area.
"Code is a bright spotlight" illuminating any funny business seems like where we are at today. And that could make all the difference.
At least the discussion has started towards this road. 2015: the year of FI's into blockchain. 2016: the development of the more complicated crypto theses.
https://steemit.com/ethereum/@justinoconnell/how-blockchain-loyalists-won-in-the-battle-for-ethereum
I would love to see an article from you on autopilot ethics and how the field has evolved since your days at SAIC to now. We really are on the forefront of machine ethics and it seems prudent to discuss what the guard rails need to look like and how we know they are actually working.
I consider myself so challenged. :o)
On the heise.de forums, someone commented on an article on the DAO trainwreck with the simple brocard pacta sunt servanda!
I replied by quoting from the "Bürgerliche Gesetzbuch" ("Civil Code"?), § 133:
So the problem is much older than smart contracts, and even a bureaucratic, complicated and technically precise law system as that of Germany insists that code is not law - the true intention it expresses is what matters when arbitrating.
I have no business with Ethereum, and have not followed the whole debate, and I take no sides.
So it seems to me that from a "code is law" perspective of trust, those who insisted that the code be followed and the chain not be hardforked to bail out everyone, were completely right. From now on, it must always be feared that a smart contract will be nullified when sufficiently many and sufficiently powerful players decide half-way in they do not like the outcome.
The community should have learned their lessons and make sure to write less stupid smart contracts from then on and continued on the way of the Classic fork.
I imagine it is quite hard to de-bug a smart contract before it goes live or to anticipate all exceptions, so leaving the role of judge, jury and executioner to a machine (Garbage in - Gospel out) does not seem a good idea at all. It can only serve as a tool to ascertain whether the parameters are right and the conditions are met; the final decisionmaking should still be with accountable beings with heart and brain.
Great write up. I've been really fascinated with the morality of artificial intelligence and put together a pindex board with some stuff I've been reading/watching along with a post here on Steemit about it. I think the automation that's coming will force humanity to figure out what we want morality to actually be. More than that, I think superintelligence (if we survive it) will help us understand ourselves and the morality we want. It will be able to analyze utility at levels we can't current conceive of. All it needs is mass amounts of data, super fast computers, and more people like Nick Bostrom and Ray Kurzweil to continue innovating and helping us think through the challenges we're going to face.
Know thyself indeed. Humanity has to figure this stuff out well enough to write it into code.
Haha that was fun to read
@lukestokes If we survive it is an if I wish we would be smart enough to avoid. But I doubt we'll be that smart. I'm mostly convinced we'll all die because some asshole creats an AI and 'turns it on' and it goes Skynet and kills us.
If you're seriously interested in this stuff, give Nick Bostrom's Superintelligence a read. That's the best thing I've seen on the topic so far. I also enjoyed The Master Algorithm by Pedro Domingos. If you're curious on the morality of A.I. stuff, check out this post which links to a lot more info.
Really nice analogy! I wonder how many would stand next to and argue with someone hacking the software in a bomb device that code is law. I mean how much should you care when faced with your own death?
Umm yeah you should censor the execution of a piece of code in order to save human lives that will die if you don't. Failure to do so makes you culpable in a murder.
Seriously, WTH?
And this is exactly why I respect the DAO Hacker. Wrote a bit about this subject here.
https://steemit.com/anarchy/@kyriacos/why-i-respect-the-dao-hacker-but-not-the-dollarvigilante
Great view on the unpleasent DAO code execution. Digital currencies will someday be a polished marvel to reminisce over pioneering
Hi, could someone help me out with a little money, haven't made with much money with Steemit, thanks.
Interesting read. Thank you!
Code is not law. In fact, code rules the program, but law rules relationship between people. And remember: there are always people behind computers and their codes. Nobody can avoid being reached by law, even behind computers.
thanks for sharing this material. I like what you posted. if you want you can visit my blogg you. Thank you
Extremely well said, and an inspiration to better things in the world of smart contracts. Joseph Campbell would also point out that justice is blind for a reason. Our human perceptions are notoriously inaccurate, and easily swayed by emotion and peer pressure.
"where it's difficult to model intent programmatically, one can still develop a domain specific language to capture it for humans to arbitrate" This captures the right path, I think. We can build in a method to ensure intentions are carried out in a lawful manner, while maintaining a ledger that documents our errors for correction, rather than attempting to erase them and rewrite history.
Excellent article. A topic I'm very much thinking about as I touched on, on my video.
https://steemit.com/ethereum/@juansgalt/6-ironies-and-myths-of-an-ethereum-drama
I liked this a lot. The one "pile-on" I would add or amplify is the whole "code is law" perspective -- I'm actually going to argue out of both sides of my mouth on this one. The for "code is law" perspective is the recognition that of that famous cliche that 9/10 's of law is based upon possession. "Code is Law" is basically a rework of that. If the "code" is the law then when things are perceived as having gone against the user of the "code" -- it is up to them (the agrived) to make their case -- the "code" is not going to advocate for them unless explicitly defined or "coded".
But the idea that you can circumvent laws based on a "I-coded-it-thats-the-way-it-is" paradyme seems a little over the top. I can definitely see that the legal side of the house (current law) will have to propose some mechanisms to legitametly provide relief to poorly written or bad "code" code is law apps.
Ethereum - Code is law until a large enough group needs a bailout.
Correct me if I am wrong but didn't Satoshi Nakamoto invent bitcoin because banks were getting a bail outs? Nah that was just the article he mentioned in the genesis block.
The problem with our current laws is they are interpreted by humans who decide the meaning and can change the meaning or just outright ignore them.
Example:
**State shall enter into any Treaty, Alliance, or Confederation; grant Letters of Marque and Reprisal; coin Money; emit Bills of Credit; make any Thing but gold and silver Coin a Tender in Payment of Debts; **
The above law could be interpreted as excluding the federal government or just ignored. The consequences of the bypass the law resulted in average people suffering financial repression as they are unable to just save without gambling on investments or speculation.
I look forward to our new monetary paradigm where savers are rewarded and debtors are punished, only binding code can make that happen and it has with bitcoin.
Just my 2 Sats worth.
Too much fake politics here hh