According to Roz Ho (a Microsoft VP) at t-mobile.com/sidekick (10/15/2009 1:00 AM PDT):
We are pleased to report that we have recovered most, if not all, customer data for those Sidekick customers whose data was affected by the recent outage.
We have determined that the outage was caused by a system failure that created data loss in the core database and the back-up. We rebuilt the system component by component, recovering data along the way. This careful process has taken a significant amount of time, but was necessary to preserve the integrity of the data.
Corporate Vice President
Premium Mobile Experiences, Microsoft Corporation
Note that updates to that link are replacements, not additions or edits.
Credit for lost service and data
Previously [10/12], we were told:
In the event certain customers have experienced a significant and permanent loss of personal content, T-Mobile will be sending these customers a $100 customer appreciation card. This will be in addition to the free month of data service that already went to Sidekick data customers. This card can be used towards T-Mobile products and services, or a customer’s T-Mobile bill. For those who fall into this category, details will be sent out in the next 14 days – there is no action needed on the part of these customers. We however remain hopeful that for the majority of our customers, personal content can be recovered. —Dan, T-MOBILE STATUS UPDATE ON SIDEKICK DATA DISRUPTION, MON., OCT. 12 , T-Mobile Forums
Fortunately, I’ve not lost anything that I’m aware of — on my phone. I already knew not to reset my phone or remove the battery. Whether my “stuff” has been recovered on the servers, I don’t know. Nor do I have any idea how to interpret “significant” or “fall into this category”.
Dogfooding or Sabotage?
Earlier speculation [10/12] at AppleInsider, based on “insider” information, suggested that it likely was “dogfooding” or perhaps even sabotage:
Additional insiders have stepped forward to shed more light into Microsoft’s troubled acquisition of Danger, its beleaguered Pink Project, and what has become one of the most high profile Information Technology disasters in recent memory.
The sources point to longstanding management issues, a culture of “dogfooding” (to eradicate any vestiges of competitor’s technologies after an acquisition), and evidence that could suggest the failure was the result of a deliberate act of sabotage.
Negligence or Incompetence?
Despite this being from the Apple camp (and my own biased opinion against Microsoft), I think that upgrading without adequate contingency planning, along with inadequate operating procedures, is likely what really happened:
Microsoft’s accountability in supporting its acquired Sidekick support obligations with T-Mobile was also shirked. The source stated that “apparently Microsoft has been lying to them [T-Mobile] this whole time about the amount of resources that they’ve been putting behind Sidekick development and support [at Danger] (in reality, it was cut down to a handful of people in Palo Alto managing some contractors in Romania, Ukraine, etc.). The reason for the deceit wasn’t purely to cover up the development of Pink but also because Microsoft could get more money from T-Mobile for their support contract if T-Mobile thought that there were still hundreds of engineers working on the Sidekick platform. As we saw from their recent embarrassment with Sidekick data outages, that has clearly not been the case for some time.”
That indicates that Danger’s high profile cloud services failure didn’t occur in spite of Microsoft’s ownership, but rather because of it. [emphasis mine –loh] This has led observers to question the company’s commitment to its other cloud services, not just Windows Mobile MyPhone, but also the Azure Services Platform of cloud computing efforts that the company has had on the drawing board for years. Azure is designed to allow third parties to build applications that are dependent upon Microsoft’s data centers.
I imagine one or more people, not necessarily the most responsible, are out of a job.
Microsoft has taken another black eye. This will bleed over into their other hosted services, current and planned. They’ve tried to distance themselves by saying that Danger was a separate unit, but they bought them over a year ago, so they “own” the issue, regardless of how much they wiggle.
T-Mobile I see more as a victim; I’m not sure how much blame for this they should share. In any case, they have halted sales of Sidekicks until this is resolved, many customers are jumping ship, and their reputation will suffer, no matter who is to blame.
Personally, I like the Sidekick despite its having a “for kids” reputation; it has an excellent keyboard, plenty of apps for what I need, and relatively inexpensive unlimited plans. For now, I’ll stick with it — surely MS/Danger will be particularly vigilant to prevent further disasters — but whenever my contract comes around, I too may switch to something else. Again, it’s back to my anti-Microsoft bias, but I’ll never trust the service like I did before.