Friday, December 31. 2010
Well, first of all, I suppose I need to have at least one blog entry per year, so this one just makes it.
For the last few months, I've been putting more and more data structures into MongoDB instead of MySQL, and I just put a task into the backlog to move everything from MySQL to Mongo.
I've worked with relational databases since 1989. Moving to an RDBMS from the indexed filesystems on the mainframe was a huge leap forward. It represented the start of the shift from overnight batch processing toward realtime data. For those twenty years I've been reverential toward the database component of the apps I've architected. I've spent significant upfront time putting together highly-normalized layouts with the goal of limiting schema changes as the product concept matures. Having had to do database conversions of various types, I've always preferred code conversions, so the idea is to overdesign the database as much as possible, and let the code catch up.
So, last year the talk around NoSQL really starts to heat up. It just doesn't hit home for me, though. Why should I change my strategy of a highly organized data structure and switch to a primitive key-store with no JOINs, no referential integrity, and no strong entity-attribute model? Sure, I can see some nice two-dimensional structures which would benefit from a high-speed data store, but that's what I use memcached for, so what's the big deal?
I decided to attend MongoNYC last May. It was convenient, it was cheap, so, why not?
I can't remember which session I was sitting in. But during that session, it hit me on the head like a licensing renewal bill from Oracle. This is an Agile development tool. If one of the important ideas behind Agile is to get a meaningful demoable feature in front of a customer or feature owner as soon as possible, then what am I doing spending all this upfront time designing a database with a dozen tables just to represent the customer contact who receives our invoice? Why can't the Customer record (document!) just have a list of contacts? I don't need a Structured Query language to assemble all the pieces of the document because I never tore the document into pieces in the first place. I can start writing the code, dump the data into a document, and get my prototype up and running.
So fast forward 6 months, and NoSQL gets more and more ink every day. Some well-respected MySQL people are still asking the wrong question. I know that performance has been a potent motivator behind the growth of MongoDB and the other NoSQL implementations, but the big deal here, folks, is development. For the last decade the true costs of running software have been tipping drastically towards product development costs and away from hardware and systems software costs. Unless of course, you're Twitter, and you want to handle 50 billion text messages per minute. But for those of us just trying to build useful apps, it's pretty cheap to throw money at performance issues. Don't get me wrong, I love the speed that Mongo gives me. But what's going to hit my bottom line is that I can get features out the door without an environmental impact statement for the database.
And, finally, I'm just having fun using MongoDB. Yup, I'm still in this for the kicks.
Tuesday, September 16. 2008
Microsoft and Cray have announced a new supercomputer that runs Windows.
I think this is a pretty good indicator of the minimum hardware requirements for the next version> of Windows.
Wednesday, April 9. 2008
Once again, I've been told that an IP address can only have one PTR resource record.
We got a new Internet connection in the office, and I requested 2 PTR records for an IP address on that connection to provide reverse address lookup. Can't have email rejected for lack of a reverse IP entry. After being ignored by Support for 2 days, I sent the request to the Account Manager, who said the request was rejected because I had 2 hostnames for one IP address, and that's not allowed.
I've heard this almost every time I've set up a new Internet connection since 1996. I can't believe there are techs who still think this is true.
From RFC2181 (http://www.ietf.org/rfc/rfc2181.txt):
10.2. PTR records
Let's go folks. Just because Jon is dead, it doesn't mean that RFCs don't matter anymore.
Thursday, March 22. 2007
Well, it's time for me to start hiring again. And slowly, despite years of experience, I convince myself that it's OK to post jobs on craigslist and dice and to allow the job posting pages on our website to be indexed and scraped. And I hold my breath. Because I don't have a database backend for the application process yet (number 43 on the list of projects that were due yesterday). Because my Inbox is about to be flooded with applications. Some of them might actually be from people with a glancing, occasional relationship to the requirements in the posting. Some of them might even be within commuting distance of our offices, although most will be from a minimum of 4000 miles away.
Please, please don't start thinking that this is some anti-immigration rant. The only person I ever met whose family has been in this country for more than 150 years was Sue S., and I think they were the advance scouting party for the Mayflower. Her family had houses (there were a bunch of them after 350 years in the same town) with 5 foot high ceilings. Immigrants are the core, the backbone, the vitality of this country. If we want to end up like Europe, as a bunch of mouldering, inbred, half-wits waiting for Doomsday, then the current trend against immigrants is a brilliant move.
Sorry, I go off like that sometimes.
Anyway, it's not about where they're from, it's about where they are, and whether that location makes it even slightly possible that they could show up in the office for more than 2 hours a day a couple times per week. Oh, yes, I'm a dinosaur, a relic of ages past. I have heard that with this modern Internet thing, nobody needs to go to work anymore. We can all sit at home, jacked into a computer and it's like the whole world is one giant virtual office.
What a load of crap. Aside from the fact that I don't want to create a race of eyeball-radiated, anti-social, speechless robots who never have direct human contact, people work better when they are in somewhat close proximity. They talk, they enjoy each other's company, they gain a real sense of community, and are motivated to consider themselves part of the group.
Gee, there I go again.
The job posting says we want people with multiple years of commercial, hands-on PHP and MySQL experience to work full-time, on-site in our Manhattan office. So naturally, I get resumes from Romania, Dallas (not sure which is farther away), India, and Canada from people with no experience in programming, or extensive experience in VB and Access. Some of these are readable, thoughtful applications. I try to turn them down politely. Most however, are illiterate (aside from the language differences in, say, Texas), and don't come close to responding to the instructions in the posting, or have 'firstname.lastname@example.org' as the From: address. They're just awful as attempts to get me interested in the human behind the email.
It's very frustrating. It shows how incredibly broken the whole hiring process is. This should be a time when you meet people who will be an asset to your organization, where you make the process of selling your company as a place to work a joy and an exciting experience for the applicant. Wait. You don't think that the hiring process is a selling process? You sit back in your web and wait for your victims to beg appropriately for the privilege of working for you? Wow. First day on the job must be a blast with all those beaten, humiliated new employees shuffling in to their drab workplace, dreading first contact with the Dark Lord of hiring.
The person/company/extra-terrestrial invading army who can fix this horrible, demeaning, counter-productive morass will change the world. Too bad I won't live to see it.
Monday, September 25. 2006
I came across Aboutus.org recently. I like it. They spider sites and pick just the About Us page.
Since it's a Wiki, you can also edit your page. So if the spider picks up the wrong About Us page, or there's more that you want to say, sign in and make it happen.
Here are my favorite AboutUs.org pages:
LinkExperts Link building ad network.
TextLink.com The LinkExperts blog.
Geody.com Look around at your planet.
BollywoodSARGAM The intersection of East and West.
Got a site? Go see if you're listed.
Sunday, July 30. 2006
Thanks to Joel Spolsky for the scan of the Traveler's Insurance ad in Inc. magazine:
Forget the geek stereotype stuff. Like the rest of humanity, you can find techies who fit stereotypes and techies who don't.
What matters here is that Traveler's is promulgating the idea that external threats are where organizations should direct the bulk of their security efforts (and, of course, increased insurance spending).
The threats come from the inside, people! Not the outside. Laptops gone missing (for some reason loaded with sensitive or confidential data), disgruntled employees, gossips, the complete and utter inability to secure media transformations (paper to disk to screen and back to paper again...), MS Outlook, security policies which focus on making legitimate tasks hard by forcing employees to jump through hoop after hoop of red tape, MS SQL Server, being unable to flexibly apply policies instead of using broad brushes to make everyone's like more difficult, and on, and on...
Whew! I feel better now. You can put away the needle with the tranquilizer. OK, yes, there are external threats, but they are much more readily identifiable, and easier to secure against.
There's also a slight matter of exactly what and how the insurer is actually underwriting. Read the contract really carefully, and you will find that unless you've been documenting every last sub-atomic particle of how you secure your premises, your systems, your operations, and your data, and that those efforts conform to some bizarre idea of security developed by an insurance industry task force, you ain't gonna see dollar one from that policy.
I am gonna get me a pair of red socks, though.
Wednesday, July 26. 2006
Back in February I told you about World Community Grid. I explained how the computing power that sits almost 90% unused on the PCs you have at home, at work, and in school could be used to save lives. And you haven't done a thing about it!
Since May, the total amount of computing time per day at World Community Grid has plummeted from 130 years to less than 90 years. It should be increasing, not decreasing. I'll say it again. There are lives at stake here. This is something that costs you nothing, nothing! It is the greatest tool to assist scientific research in history. That's no exaggeration. This is as if every researcher in the world was all of a sudden given their own university with 10,000 research assistants.
The knowledge needed to understand disease and develop effective treatments and PREVENTION lies in the ridiculously overpowered machine that you bought to read email or make your mind-numbing Powerpoint presentations. You're not sharing it. Remember, sharing is good!
Seriously, join this project. If you don't like what World Community Grid is doing, go to the BOINC project at Berkeley. Their software is being used on a lot of different efforts. Find something that suits you and join in.
Monday, July 24. 2006
Have you heard of linkie winkie?
No?, well neither have I.
Everybody's talking about it. See if your site has been linkie-winkie'd.
Nope, I'm not telling. You have to figure this linkie winkie thing out for yourself.
Wednesday, July 5. 2006
So, as SCO faces additional setbacks in the months to come, losing more motions
and receiving additional sanctions, losing appeals, not to mention being faced
with the Everest of evidence that their claim of any kind of proprietary rights
to Unix and it's derivatives and "children in spirit" is wishful
thinking at best, the thought of settlement negotiations arises.
So, kiddies, let's hear your suggestions for IBM's counsel. What should they
throw on the table to help SCO and friends avoid the penalties and costs they
face by losing at trial.
I'll start it off:
No costs, no penalties (to SCO or it's lawyers), not one dollar. They can stay
in business (if you can call it a business, whatever it is that they sell).
All they have to do is place ALL (I mean every last line of code, piece of
documentation, even their logo) under the GPL.
That's all. No big deal.
Friday, June 30. 2006
In her ruling on IBM's motion to strike a portion of SCO's claims, Judge Wells said that she did not consider the merits... yet. All that she was ruling on was SCO's adherence to discovery rules, and especially the orders she issued on discovery to both sides.
The time has not yet come. the Judge has said,
To speak of so many things.
Of methods and concepts,
And how The SCO Group sings.
But the time will come for legal precepts,
And whether pigs have wings.
That's when the rest of this nonsense will be thrown out.
(sorry, mr. dodgson)
Wednesday, May 31. 2006
If you can't be bothered to visit this site every day to figure out if I've let another 2 months go by without a post, or if RSS/Atom feeds haven't reached your level of conciousness yet, there's an alternative.
Down on the right you'll see a place where you can enter your email address. Submitting your address will get you an email feed of new and updated articles. Isn't it wonderful and amazing?!!??
FeedBlitz powers this little feature, and appears to be winning out over Bloglet which has been accused of less than reliable service.
Tuesday, February 7. 2006
No other technological breakthrough has demonstrated the power of individuals more than grid computing. By donating your unused computer time, you can begin to change the world for the better.
That's what is says on the World Community Grid website. And they're right!
Your PC at home or at work is in use MAYBE 5% of the time. So, assuming you have an ancient Pentium III processor runnning at over 1 BILLION Theoretical Operations per Second, each day that goes by sees 82,080,000,000,000 potential calculations NOT being performed.
Now, suppose that you could install a piece of software that knew when you weren't using your computer. It would communicate with a central source, download some work to do, use a few trillion of those available calculations, and report the computed results back to home base.
Now suppose that this is done every day by hundreds of thousands of computers worldwide, and that the work being done allows scientists and other researchers to avoid hundreds of thousands of hours of trial and error research. IMAGINE the things that might be achieved!
The World Community Grid offers a screensaver which will use the available, otherwise unneeded computing power of the machines that sit idle almost all day long while you sleep or are at work or anywhere to develop new anti-AIDS treatment, to analyze the human genome for potential disease prevention, or for any of the other projects that will utilize this platform.
Get this screensaver now. Think of the lives that are lost because of every second you delay!
Seriously. This is a GOOD thing. It's not a security risk, it's not spyware. It doesn't impact you when you're using your PC. Do it. You have no idea what you might be accomplishing.
If the projects at World Community Grid aren't to your liking, break out your favorite search engine and look for "screensaver distributed grid computing".
Monday, January 30. 2006
The New York PHP Users Group has announced that it will hold a PHP Conference in New York City in June. NYPHP is not the typical techie user group where a bunch of spotty geeks sit around telling war stories. NYPHP has monthly presentations with substantial content on a variety of topics, offers classes, not only in PHP syntax, but in practical implementations as well, including security issues.
To find out what PHP represents as an full-strength solution to enterprise-class problems and as an alternative to that bloated, (see also) marketing hype whose name we won't mention, attend this conference.
-- Sign-Up Now To Lock in Your Early Bird Discount
-- The Call For Papers is Now Open
-- Official Press Release is Available
Saturday, October 29. 2005
Brian Coyner has an entry on the O'Reilly weblogs about bad database design leading to even worse code.
First, let me say: Here, Here! and Huzzah!
When you design an application, you must spend time over-designing your database before you write any code. Assume that every requirement that you have been given will change before the bits are dry in your inbox. Assume that every time your business analyst or user liason or boss said "That's as far as we will ever go with this product", they're wrong (or lying, your choice). Modifying code is a thousand times faster, easier, more reliable, and cheaper than a database modification and conversion. Studies have indicated that maintenance consumes up to 85% of the software budget. My experience has been closer to 98%. If you can move effort from maintenance to development, you move expenses from the cost center to the profit center.
Brian has a little example in this article, though, which I just have to jump on. He promotes the use of non-mutable key values for tables. That is, the key should not contain any business information which could change. I agree 100%. The example shows that by moving an account number from a key column to a non-key column, only an update of the non-key column is required.
What is a column update? It is the destruction and loss of data by replacing it with new data. What happens when someone (auditing, marketing, business intelligence applications) go looking at this customer or vendor? They see a history of events (payments, orders, disciplinary actions) against the current account number. It may be that none of those events pertain to that account. They occured when the value in that column was something different.
So when you sit down to overdesign the database for your next application, think about how you will preserve and track all the daily little changes to your application's datastore. That store is not simply a snapshot of the current state. It must represent the changes that have occured and the timeframes in which they occurred.
Wednesday, October 26. 2005
A graphic (see below) on the FEMA website seems to show that the next step after Preparedness is Disaster.
Jeremy Zawodny has an entry that talks about this, but I think Jeremy is slightly off when he describes disaster leading to response, which leads to recovery, etc.
If you look carefully at this image, you will see that, except for Preparedness, in each of the phases of this diagram the arrows miss the next phase. Preparedness is the only phase which leads directly to the next one, which is Disaster.
Maybe the artist knew exactly what they were doing.
Garr Reynolds takes all the fun out of this by actually doing a professional analysis of the image and it's failure as a graphical presentation of FEMA's mission.
Great quote, though: "Powerpoint should do no harm" (Edward Tufte)
(Page 1 of 1, totaling 15 entries)