Duping Incident (Post Mortem)

Hi all,


On December 4th, I pushed a new update which included a new set of features. One of those features was the Bank. The Bank gave players the ability to store large quantities of pets & diamonds in real-time with friends. This was a super cool feature in hindsight but it introduced a big problem. Long story short, two vulnerabilities were found & abused which allowed duping (duplicating) both pets and diamonds. This has since been patched.


What happened?

The short answer: two technical oversights. These technical oversights were abused knowingly by players to duplicate items.


The first technical oversight was quickly found, verified, and fixed. It was just a mishandling of data between the player's inventory and their bank.


The second technical oversight was a lot more complicated and took almost 2 days to verify. The documentation for reading and writing bank data that we used on our database was outdated/deprecated. There was a very small time window where data would return cached instead of the correctly up-to-date version. Unfortunately, there was no way to catch something like this until after the update was live.


(This was the culprit for those interested. This is in MongoDB's official documentation. For some reason it wasn't correctly marked as deprecated.)



What did I do about it?

A couple things.


- Duped pets will be deleted automatically. This will extend to banks. Unfortunately, this includes innocent players. This wasn't an ideal solution... and it definitely hurt to do... but it was either all or nothing. It's better to do this now instead of waiting and making the decision later.


- Accounts made specifically to horde these pets/diamonds (stashes) have been banned.


- Users caught abusing these exploits have been wiped/banned including their associates and alternate accounts. This included lots of blackmarket ring-leaders and storage accounts caught in the process.


- Over 1,000 malicious banks have been wiped.


- Players with glitched Diamonds have been reset to 0.


- A system is now in place to monitor and log any unusual activity, duped pets, and more.


- Duped pets will no longer be possible from this point forward. I say this with confidence.



Going forward


I want to stress that I care deeply about the game economy and players. Pets shouldn't be devalued overnight because of a technical oversight or exploit. I put a lot of hard work and passion into creating a thriving economy and I hate when these things happen as much as everyone else. It ruins the fun for everyone. It sucks.


I will continue constantly monitoring players and ensuring, to the best of my ability, these things don't happen again. A lot more systems are now in place and I'm taking every precaution from this point forward.


I plan to introduce more ways to earn significantly more Diamonds to alleviate the inflation from this incident and keep things fun and fair for everyone. Hopefully things level out like they did last time within ~2 weeks.


Thanks for understanding.


Christmas Update on next Saturday!