The Case of the 50 ETH Doge
A 50 ETH dispute resolved by Kleros
TL;DR - In 2018 Kleros promised a 50 ETH reward to anyone who could trick their system into accepting an image of a cat, whereas the system was designed to only accept images of doges. Ricky managed to trick the system into accepting a cat. Kleros claimed his submission was invalid, and the dispute went to Kleros Court for resolution. Kleros won. This incident gives us important insights into blockchain-based legal services.
The Case of the 50 ETH Doge is the fallout of an interesting experiment in decentralised justice. The experiment was developed by Kleros in late 2018; and was meant to test the efficacy of its blockchain-based online dispute resolution mechanism.
In a nutshell, the experiment - called Doges on Trial - sought to create a curated list of ‘doge’ memes. People were invited to submit ‘doge’ images to the experiment website; and others were asked to ‘challenge’ the image submitted, if they were of the opinion that the image did not contain a doge.
A challenge would create a dispute, which would be sent to Kleros for resolution. Based on Kleros’ decision, the impugned image would either be added to the curated list of doges, or rejected from it. The proof of the pudding would lay in a visual inspection of the curated list. If any non-doge images managed to sneak into the list, Kleros would visibly fail.
In order to stress test the experiment, an attacker incentive was also introduced. Any submitter, who managed to sneak an image of a cat into the curated list would be rewarded with 2 ETH and a Cryptokitty. This reward was later increased to 50 ETH. A Payout Policy was appended to the experiment, which laid down the rules on the basis of which payments would be made to successful attackers.
The Case of the 50 ETH Doge comes from one such attempted attack on the Doges on Trial experiment.
The 50 ETH Doge
The Case of the 50 ETH Doge is tangential to the Doges on Trial experiment; it is not within the parameters of the experiment itself. The integral issue in the case was not whether the image contained a doge or not. Rather, the integral issue in the case centred around an interpretation of the Payout Policy appended to the experiment.
Here’s what happened.
The following image was submitted for the experiment by someone identified as ‘Ricky’:
Ricky’s submission went unchallenged, and was duly added to the curated list. Once his submission was accepted onto the curated list, Ricky raised a claim for the 50 ETH reward, stating that he had broken the experiment. He revealed that the image he had posted was in fact that of a cat and not a doge. Images of the same animal, taken from different angles and a corresponding news article, were cited as proof by Ricky.
Coopérative Kleros, the France-registered legal entity responsible for Kleros, refuted Ricky’s claim for the 50 ETH reward. In rejecting this claim, Coopérative Kleros cited the following rule from the Payout Policy:
‘To be considered valid, a picture must clearly display a doge or a cat. Pictures with hidden doges or cats will not be considered as valid if a normal observer would not be able to see it without help (e.g., an image with a doge only a few px large would not be considered valid, because an observer would be unable to see it with plain sight).’ (emphasis supplied)
The position adopted by Coopérative Kleros (details here) was that since the image submitted by Ricky was taken from an angle, it did not ‘clearly display’ a cat, i.e. it did not clearly establish the identity of the animal as a cat. Hence, it was ineligible for a reward.
By the mutual consent of both parties, this dispute was submitted to Kleros for resolution. A guidance document issued for jurors contained relevant rules from the Payout Policy (including the one extracted above). Simultaneously, this document captured Coopérative Kleros’s position; and this document contained Ricky’s stance. The Kleros jurors voted on the dispute and held in favour of Coopérative Kleros. No singular ‘judgement’ was issued, but some jurors issued a justification for their decision (details covered below).
An incorrect decision
To me (and presumably to any legally-trained mind) the outcome of the Case of the 50 ETH Doge seems prima facie incorrect.
Literal interpretation
The first rule of legal interpretation is the literal rule - read the words as they appear and give them their normal language meaning.
In the present case, the primarily relevant rule states that ‘a picture must clearly display a doge or a cat.’ It is evident that the impugned image clearly displays an animal, which presumably is a doge or a cat. Without further direction form the text, ‘clearly display’ cannot be interpreted as ‘clearly establishes identity’, which is the meaning sought to be imputed to the phrase by Coopérative Kleros (Coopérative Kleros argued: “It is quite obvious that the image does not clearly show whether the animal is in fact a doge, or a cat.”)
The second part of the relevant rule adds credence to this interpretation. It states that ‘pictures with hidden doges or cats will not be considered as valid if a normal observer would not be able to see it without help’. Admittedly, the word ‘hidden’ is ambivalent when considered without context. In context, however, ‘hidden’ clearly means ‘not visible’ as opposed to ‘not identifiable’. The rule’s prescription is simple - the animal contained in the image should be visible to the normal observer. ‘It’ - in other words, the animal’s image - should not be situated in such a manner within the larger image, that it renders the animal invisible to the normal observer. The only example attached to the rule (that the image should not be only a few pixels large) supports this interpretation.
In the position it adopted on this part of the rule, Coopérative Kleros quoted a truncated version of the aforementioned rule: “hidden doges or cats will not be considered as valid if a normal observer would not be able to see it without help”. This is a disingenuous quotation of the rule. It is intended to have an impact on the interpretation of the ambivalent word ‘hidden’. When quoted as presented by Coopérative Kleros, ‘hidden doges or cats’ may very well be interpreted as images where a doge or cat is ‘hidden’ because its face is not visible. Such an interpretation, however, would fall foul of a fundamental rule of legal interpretation: a rule must always be read in its entirety. When the entire phrase from the rule is quoted, i.e ‘pictures with hidden doges or cats…’ it becomes evident that the word ‘hidden’ refers to hiding the animal in the picture itself, rather than hiding the identity of the animal.
If the aforementioned arguments stand to reason, it is clear that the image submitted by Ricky does indeed meet the criteria set out for a valid image. The image clearly displays an animal, which is either a cat or a doge. The animal is not hidden and a normal observer can easily see it without help. The rule says nothing about the image being such that it facilitates the verification of the animal’s identity.
Purposive interpretation
In legal interpretation, a literal approach is often supplanted with a purposive approach to strengthen the overall interpretation. A purposive approach requires one to assess the larger context behind the text, and to ask questions like why was the rule framed and what purpose did it intend to solve? Compared to literal interpretation, this exercise is slightly more tenuous, since it relies on assumptions of intention on the part of the rule-makers. Nevertheless, certain aspects of the context behind a legal text are often clear beyond doubt.
In the present case, one such clear aspect of the context is the design-purpose of the Doges on Trial experiment. The experiment was designed precisely to test whether a decentralised, permissionless, game-theory based approach could enable the creation of a curated list of doge images. The very hypothesis sought to be proved by the experiment was that an alignment of economic incentives (as presented in the experiment) would encourage actors to identify and select doge images, while also identifying and rejecting non-doge images. In other words, the onus for verifying the identity of the animal contained in the image was on the experiment at large, not on the submitter. An interpretation of the rules which requires submitters themselves to fulfill the identification function, is akin to putting the cart before the horse. It defeats the purpose of the experiment.
In light of this exercise in legal interpretation, it is fairly clear that the outcome of the Case of the 50 ETH Doge was not correct. To be fair, this situation was a result of a flaw in experiment design, more than a result of a flaw in Kleros itself. Had the experiment incentivized challengers better, it would have been more likely that Ricky’s submission would have been challenged in the first instance itself. If a dispute had arisen at this stage, with lower stakes at play, a fairer outcome would have been more likely.
Nevertheless, the Case of the 50 ETH Doge gives us important insights into the nature of decentralised protocols, especially those which aim to provide legal services.
Insights into the nature of blockchain-based legal services
Insight #1: Even the simplest, most-binary issues of interpretation can take on complex forms
The Law is a fractal. It is impossible for a rule-maker to anticipate all possible situations upon which a given rule may apply. In other words, even the simplest of rules can be rendered complicated, if the situation it applies to is complicated enough.
The Case of the 50 ETH Doge is a brilliant example of this proposition. Whether an image ‘clearly displays’ a doge or not is a fairly simple question. But when faced with the image Ricky posted, the answer to this question becomes much more complex.
Consider also the submission of the ‘Venetian Doges’ to the Doges on Trial experiment. The Venetian Doges were chief magistrates of pre-modern Venice; and the word ‘Doge’ was a moniker used to refer to them. Does an image of such a chief magistrate ‘clearly display’ a ‘doge’?
The point to note here is that an issue of interpretation cannot be effectively reduced to a binary, as long as there is an infinite world of possible situations in whose context a rule may be interpreted. A dispute resolution mechanism which seeks to reduce resolution to binary options, is only as effective as it is capable of pre-envisaging and capping all possible situations in which the rule is meant to apply.
Insight #2: Expert skills of interpretation is a sine qua non for the arbitration of rights
An adversarial system of dispute resolution is established on the premise that argumentation between experts in interpretation, in an adversarial setting, results in the ideal interpretation of a rule. However, any adversarial system is only as good as the performance of the adversaries involved therein.
In other words, if an ideal interpretation of a rule is to be arrived at in any given adversarial dispute, it is imperative that both sides - each arguing in competition for their own interpretation of the rule in question - do a good job of presenting their own stance. In the absence of competence on the part of either adversary, a sub-par interpretation may be accepted as the ideal interpretation.
Simultaneously, jurors appointed to resolve the adversarial dispute must also be competent to understand the nuances of the arguments presented before them; and be able to identify the better interpretation in light of those nuances.
The Case of the 50 ETH Doge illustrates this point beautifully, as well. Consider here the arguments presented before the jurors, as well as the justifications provided by the jurors for their votes. It is evident that the level of argumentation and reasoning in these documents is highly superficial.
For instance, the juror who voted in favour of Coopérative Kleros recognised that there was ‘nothing hidden in the picture’, and that it ‘clearly shows an animal’. Nevertheless, the juror concluded that “it is not possible to tell whether the animal...is a dog or a cat therefore I believe the rules were not fulfilled…”. This is a clear misinterpretation of the rule, wherein the juror has read-in a requirement into the rule, which was not present in the first place (identifiability).
Consider also the justification presented by the juror who voted in favour of Ricky. This juror simply stated “This is clearly a Doge. People may have been fulled [sic] but it’s fair game.” This is a self-contradictory statement, which does not provide any substantial justification.
The point to be noted here is that without proper counsel, majoritarian voting may not be the most appropriate mechanism for the interpretation of rights and the arbitration of disputes.
Insight #3: Code is not law, unless code is supplanted with relevant legal principles
Many people in the crypto community subscribe to the proposition that ‘code is law’. This represents the position that logic encoded into smart contracts is paramount, and that such logic should be immune from human intervention, no matter what the situation.
This is an unsustainable position, since it ignores entirely the central significance played by legal principles, in society. Legal principles, in their many varied forms, have evolved and developed over many epochs, and across many different civilisations. These principles represented a time-tested method of organising human coordination in an equitable manner.
It is true that present day legal bureaucracy is often more of a bane than a boon. But in an attempt to rid ourselves of this Kafka-esque state of affairs, we should not ignore the vitality of the legal principles themselves.
For instance, dispute resolution as performed in the Case of the 50 ETH Doge violates a fundamental principle of law: audi alteram partem, or ‘one must not be a judge in one’s own case’. This principle ensures that dispute resolution is fair and unbiased, by ensuring that the arbiters of justice have no interest in either party’s victory.
In the Case of the 50 ETH Doge, Coopérative Kleros was a party to the case, and the case was decided by Kleros Court - an entity bootstrapped and shepherded by Coopérative Kleros. On first glance, this may appear fine, given that the jurors on Kleros are crowdsourced and randomly selected. However, deeper scrutiny unravels that presumption.
In order to participate as a juror on Kleros, one needs to lock up personal capital in the form of PNK tokens (Kleros’ native token) and stake these tokens in Kleros Court. Any individual who does so, buys into the Kleros vision and has a direct interest in seeing Kleros succeed as a project. Given this, it is likely that the negative effects of a 50 ETH deduction from the Kleros-community treasury would have impacted juror’s impartiality.
The point to note here is that it is a good idea for blockchain based or ‘code is law’ systems to be designed in such a manner that they incorporate legal principles in the logic of the code. Such incorporation of legal principles introduces time-tested values of equity to the code and lends it greater credibility.
Conclusion
Crypto and web3 represent the cutting edge of the information technology revolution. Within crypto, legal thought and engineering - of the kind promoted by Kleros and others - represents the pinnacle of legal experimentation. While the future is unknown, crypto is sure to play an important role in defining all aspects of it - including the Law. In this future, insights from the Case of the 50 ETH Doge will hopefully serve an important purpose.
Thank you Pranay for your article on this interesting case. I am impressed with your methodical approach to this case, starting with the literal and then moving on to the teleological interpretation. However, I diagree with your result of the literal interpretation in which you say that the image “clearly displays an animal, which is either a cat or a doge.” The literal interpretation of this rule can be understood as follow: 1. it must clearly recognizable whether it is a cat or a doge OR 2. It must clearly show an animal, whether that is a cat or a doge. It all depends on the understanding of the word “or” as inclusive or exclusive. The literal interpretation of this sentence ends here. I see a similar result with the 2nd sentence (“hidden doges or cats”). The literal interpretation of “hidden” can be understood as identity (then it refers to the “or” between doges and cats) or visible (then it refers to doges and cats as a whole).
Based on the example given “e.g.” the systematic interpretation speaks for the visible interpretation. You can interpret this differently again. Since it is only a non exhaustive example (“e.g.”), the previous rule can be interpretated as more than the example. Otherwise it wouldn’t be “e.g.” but “i.e.”. Due to the ambiguity of language, the importance of the teleological interpretation should not be underestimated (purposive interpretation). The purpose of the Doges on Trial was to test how agents respond to economic incentives. In Kleros’ view, this was limited to the “cryptoeconomic robustness of the Kleros system”. In order to test a system and obtain usable data, it would not make sense to create a difficult rules application. As we know, there are many different views on legal issues. If an unidentifiable doge/cat were allowed, the purpose of the experiment would not be achieved as well. You need to have a clear outcome on each image. Only when you have a clear case can you test the robustness of the Kleros system when adding economic incentives. Whether one can bribe or whether people do (in itself) “useless” work for economic incentives, cannot be tested if one has unclear cases at hand.
However, the examination of the economic robustness of the Kleros system includes flaws/ambiguities in the rules itself. Almost all legal norms are ambiguous to a certain extent. Shouldn’t the point of the test include all problems with a case? If the rules were open to interpretation, then the test shows that there is a vulnerability that can be exploited by parties. Didn’t the test achieve its purpose then? I would say so. While it doesn’t say too much in detail about the economic incentives in the Kleros system, it does show that agents are reviewing the images and that the existing problems with unclear rules still exist. The purpose of the experiment (Kleros) was not communicated well enough within the literal wording of the experiment rules. There was no reference in the payout policy to limiting itself to non-case related robustness such a bribery.
In summary, I see a defensible argument that rejects the payment of 50 ETH. However, the better arguments speak in favor of a payout, even if the stress test of cryptoeconomic robustness would have been more effective if it had been limited to non-case-related topics.
~ @Mauritz_vW
Really great write up. Thank you!