Tuesday, January 29, 2008

MySQL dumped - Oracle loaded - Whose fault is it?

The purpose of this post isn't by any way to bad mouth MySQL but rather to show the impact of lack of qualified DBAs to those who use MySQL.

I was shocked last year, when a CTO of a large company confided in me and said basically that they are so tired of searching for qualified MySQL DBAs that they may switch to Oracle. At some point, I guess the frustration alone can justify an otherwise unneeded cap-ex of such magnitude.

Today, I hear that ValueCentric, a pharmaceutical technology consulting firm based in New York, has decided to let go of MySQL in their environment and instead switched to Oracle on Oracle Linux.
"as the firm expanded and began taking on bigger clients like AstraZeneca, Roche and P&G Pharmaceuticals, company officials became concerned over MySQL's ability to keep up with rapidly growing data stores and complex quality-of-service requirements."
Now, MySQL may not be an ideal fit for many
"We were faced with some pretty big issues around performance, both in processing and on the back end, as well as reporting, where we had problems with queries returning on time and being able to handle the processing of data at night -- or whenever it came in -- in an expedient manner," Janca explained. "We needed to step back and see where we were headed strategically, do an assessment and try to remove some risk that we thought we had in continuing down the path with MySQL."
This gives the sense that MySQL cannot hold up as a company grows. However, to me, it seems like they had someone with little or no MySQL experience working on it. Or worst, their data architect did a horrible job of architecting their data and reporting environment. Or even worse, they hired services of a consultant who showed them the only way out of their issues was MySQL. There isn't a shortage of such consultants, you know.

Folks, MySQL is a tool and you need to know how to use that tool to get the job done. Just because you have been able to successfully open a tool doesn't mean that you know how to use it.

No one argues the need to have a competent DBA when one is using Oracle. However, the notion of Open Source makes people somehow believe that MySQL should run great out of the box even if the database architecture of the company sucks.

One of the biggest mistakes companies and IT managers make, IMHO, is to hire a data architect with minimal MySQL experience. They think their data can be architected by someone who doesn't necessarily has the expertise with underlying database. This can be a disastrous call as a competent data architect must possess an in-depth knowledge of the underlying database.
ValueCentric had also become disenchanted with MySQL's disaster-recovery capabilities and support services, added John LoFaso, ValueCentric's director of technical operations.
Those of you who attended my talk at MySQL Camp II, know the hell I went through on 7.5.07, when our SAN crashed leaving more than 1TB of InnoDB data severely corrupt (won't stay started even with innodb_force_recovery set to 6). For sure, I wasn't able to tap any readily-available resource to recover from this disaster. However, I was able to write programs myself to extract data from InnoDB tablespaces and the recovery rate was truly impressive. However, I realize not everyone will be wanting to write programs themselves especially in the times of extremely high stress (i.e. when disaster has occurred). This is one area that is definitely still ignored by MySQL. Peter Zaitsev recently blogged about releasing a DR tool for InnoDB however, I don't think that has happened yet.

"The support was very sketchy, and we just couldn't rely on it," he said.

Ok, since this is a somewhat touchy subject, for me and many others so I will mainly let it pass. I do however feel that you need to have the highest support plan to really get MySQL to help you. We, for one, have paid for support but hardly use it (MySQL software is so great).

I find it pretty amusing that while people use MYSQL, their budget for MySQL support is just pennies compared to when their support budget for Oracle. I believe had they spent same amount of resources on MySQL support, they would be a happy MySQL user.

One thing I will say is that we are preferring to bring in independent consultants for our next major project, though, that may/may not qualify as support to some.

So in case you are wondering why ValueCentric didn't go with Microsoft?
"because of "impersonal" support operations" and because "The goal was that we needed a partner, not an 800-number"
I believe in one thing: MySQL has a remarkable community, remarkable product, remarkable leadership and a remarkable vision so why not make the support remarkable as well? Right now, it may be great support, but it certainly isn't remarkable. And to know the difference between great and remarkable, well, you gotta read Purple Cow. (hint: they are both opposite of each other).

Going back to the shortage of competent MySQL DBAs, this is something that, I think, MySQL can artificially overcome by making their support remarkable.

However, to be fair to MySQL, the support is all there, however as with most things in life, it has a price. Ok, I seem to be in a catch 22 situation so I guess, enough for today.

BTW, thanks Ronald, for giving me Purple Cow as a gift in 2006. The book has really changed my view of thinking.


Anonymous said...

you cannot call a community when you have a crisis - and not everyone can write code to extract data the way you did

Sheeri K. Cabral said...

My company specializes in being a company's DBA -- Whether a company needs as little as 4 hours a week of DBA work, the standard 40 for 1 full-time DBA, or 400 hours a week, we've got the resources you need. :) Seriously, though, I work for them, and if that's not enough, 50+ other smart DBAs work for the Pythian Group.

May I suggest trying us out for your next project? Also, can you suggest us when people are looking for DBAs? There are so many ways in which we're better than a "normal" standard full-time DBA.