A couple things. First, DTB (rated PG-13 link). SRSLY. The high school girlfriend is NOT the one, nor does she treat you well at all. It’s gonna be over a year after you graduate, so do yourself a favor and break it off now so you can enjoy college more.
Second, study more, game less. It’s gotten ridiculous. You had straight A’s. You’ll get your first D EVER if you don’t take that network programming class seriously. And it will hurt. Your ego AND your GPA.
Finally! Hardware ain’t for you. Software is where it’s at. And data stuff. ALL the data! Someone’s gonna coin the phrase “data is the new oil”. Hey! That should be YOU! Do it.
Where the Rubber Meets the Road
Your first job is going to be amazing. You’ll spend NINE years at that company. Probably a bit too long. They weren’t very understanding when your wife was very sick and needed your help running around to doctors and pharmacies and such. But you’ll work some truly awesome people, connections that will last many years. That’s called ‘networking’ — making sure you stay in touch with those people. It will help open up future career opportunities.
Yes, I said “wife”. We’ll get to that in a bit.
Spend your 20s doing silly things, adventurous things, but focus on your career. If I had to do it all over again, I would branch out sooner, explore more opportunities earlier, and look at the world of tech beyond this little suburb. There is so much more out there. Not that there’s anything wrong with suburbia — our town is great. I love it. But it’s not a tech-job haven. So don’t be afraid of that big change, that big opportunity.
Because once you go for it, once you finally take that chance, you’ll be oh-so-much happier! Ride that wave of confidence and of learning new things. Start attending SQL Saturdays earlier, and look into other data-centric meetups and events. There is so much to learn, and never enough time.
Yes, you will find love. You will find the love of your life. The One. Your soulmate. When you least expect it. You will spend your latter 20s and early 30s having the best years of your life with her. She will be your everything. She is laughter, passion, heartache, support, grace, care, light, love, and life itself. Cherish every moment.
A question that’s often asked of grievers is, if they knew what would happen, would they go back, change anything, do anything different. In this thought experiment, even — would I tell my past self what would happen to my wife? No, I don’t think I would. For even now, as I look back on the million little moments that we shared and loved and laughed and cried. I would do it all again in a heartbeat. Our love was once-in-a-lifetime. I know that in my soul. And I carry that flame in my heart. May it never be extinguished.
Head on over to natethewriter.home.blog (yes, it’s still Wordpress, because I’m cheap!)
Yes, dear readers, it’s high time I drew a clear line between the tech/professional content and the personal. Also, I realized that I need to write more, because it’s been helping me through the grieving process. And a lot of that writing is going to be very… non-technical.
Don’t get me wrong; I need to, and will, resume my database-related blogging too. At some point. I promise! But I’m trying to honor my wife’s memory and take to heart her words of encouragement to me, when she said “You know, people always told me I should be a writer. But you should try it too; you’re pretty good at it.” I feel like she’s passed along some of her soul and spirit to me. If nothing else, at least I’ll get to show the world a little glimpse into her imagination, as I gather the little bits of sparkle that she left with me in her journals and scrapbooks. And hopefully, honor that soul and spirit by exposing some of my own.
So, head on over to natethewriter.home.blog (because I wasn’t feeling terribly creative — I know, not a good start for a so-called “writer” — but I decided to stick with the pattern)… even though I have absolutely NO delusions that I’m in any sense a “real writer”, just an amateur with the desire to express thoughts and ideas in concrete form. And follow, subscribe, stalk, etc.! Hope to see you there in the comments. ❤
Data is digital information. A database is a collection of data. And a DBA manages it all.
Borrowing from an ‘old meme’ a bit. My wife recently said I should “write something about ‘how to do databases’.” As amusingly odd as her phrasing was, I figured she was right.
What is it?
I like to start at beginning. As Julie Andrews said, it’s a very good place to start. What is a database? That’s a pretty good question. Here’s the prerequisite question: What is data? Well, as I’ve said before, data is everything. But that’s a bit of a cop-out, isn’t it? That’s my career’s bias showing through.
Data is digital information. Anything that can be quantified, specified, categorized, searched, sorted, produced, consumed, read, written, measured, and stored digitally in some fashion. Data is the digital currency of the 21st century. Data is the very reason that most technology exists — to house and transport data among producers and consumers of information. It’s the evolutionary culmination of the stone tablet, the papyrus scroll, the bound book, the printing press, the newspaper, the library, the vinyl record, the magnetic tape, the compact disc, the pocket organizer, and the telephone.
So then, what is a database? Simply put, it’s a collection of data. The simplest analogy, depending on your age, is either a phone book or your cell phone’s contacts list (which is really just a phone book, in digital form). Of course, with the latter, it’s not so much an analogy as an example — you phone’s contact list IS a database.
Fun side-note, the phone book also makes a decent discussion prop for some DBA topics like index fragmentation.
Expanding on that example. You can search and sort your contacts by several data points: first name, last name, phone #, email, notes. Different database systems have various names for these: fields, columns, properties, criteria, values. The point is, it’s all data. Or if you want to get pedantic, each one is a datum, and together they aredata.
Pedantic, me? Never.
This is what a database, or DB for short, is all about: storing data in an organized fashion so that it can be sorted, searched, sliced and diced. Building on that, a database management system is a set of technology tools, processes and programs, that are used to gather, store, manipulate, copy, move, read, maintain, back up, link together, and operate one or many databases. This DBMS can come in many flavors. I happen to specialize in one called SQL Server, a Microsoft product/platform of the ‘relational‘ flavor — so if you’re following along with the abbreviation game, that’s an RDBMS.
If you’re hungry for more acronyms, the Wiki article on ‘databases‘ has a decent breakdown of the types and history behind them.
The more data you have, the more you can do with it. Why do you think Facebook, Google, Microsoft, and Amazon are such powerful technological forces? They purposefully, systematically gather as much data as they can from every possible source, and they have become very good at organizing and managing that data to maximize its value. Amazon product recommendations are a prime (see what I did there?) example — they are generally appropriate and effective because they have “learned” from your past purchases, i.e. your historical data. This “learning” – Machine Learning, aka Data Science – is the hot new marketing buzzword of recent years, but it all still comes back to data at the core.
This is not a “bad thing” or a “scary thing” as the old media and tin-foil-hat-wearers would have you believe. Yes, it can be a little disconcerting, and yes, people and companies can abuse data in malicious ways. But the vast majority of our digital data stewards actually want to do good. They want to connect you with more people that you may know and become friends with; they want you to watch movies that you’ll really enjoy; they want you to easily navigate to your destination without being stuck in traffic; they even want to help stop global warming!
As a general business rule, we crave data because it helps us make decisions. Every time a customer buys a product, we want to measure “the 5 W’s”: who what when where and how (ok, that’s not a ‘W’, but there’s a reason for it). Notice I didn’t list “why” there — only the customer knows why, and that information, that data, is stored inside their brain. And we can’t (yet) access that data. So it’s a guessing game now — we feed the other 5 data-points into our DBMS and eventually, given some time and analysis, we can guess the Why. And pretty accurately, at that. Then, we can make a decision to “Market more aggressively to Customer Type X”, or “Have a flash-sale on Product Y”, or “Move on this hot emerging market demographic.”
So what does that make you?
Well, I’m a Database Administrator – a DBA. Which means I “administrate databases”.
‘Administrate’, less common form of ‘administer’: manage and be responsible for the running of.
Thanks, dictionary. So in a nutshell, a DBA manages data. Deceptively simple sounding, no? I mean, what can data possibly do; it’s not alive, right? Actually, if you hang around a DBA for any length of time, you’ll commonly hear the phrase “Where does that data live?” or “That set of data lives over here.” So clearly we anthropomorphize our data. Most tech professionals do that to whatever technology they work closely with — it’s human nature. Software “behaves badly”, machines “throw a fit”, etc.
But anyway, why do databases need to be managed? What can happen to them?
Developers. Developers happen. =D
I joke, as you know, dear reader; I love developers. Users ‘happen’, too — often more catastrophically. So it’s fair to say that “people happen”. But besides that, here are some common reasons that databases, and data, need to be managed.
Data can be “wrong”.
Data can either be human-generated or machine-generated. Fingers on a keyboard, or sensors on a circuit board. You wouldn’t think the latter could possibly ever be “wrong”, but both kinds are subject to error. It’s just that the level of “wrongness” is subjective and depends on who’s asking and what’s expected of the system as a whole.
Data gets lost.
Humans interact with and manipulate data, and humans make mistakes. Why do you think the Undo button became such a staple of so many computer applications?
Data gets corrupted.
Storage media (magnetic disks, silicon chips, etc.) are not perfect — they have a documented level of fault tolerance and failure rate — so we need to ensure that our data is preserved (by moving it to another area that’s not ‘faulty’, usually) past those failures. Why? Because our data is essentially “more valuable” than the hardware on which it’s stored.
Data needs to be organized.
This is slightly more subjective than the above; how and why we organize data is highly dependent on the overall intent of the systems that will interact with it. But fundamentally, if there’s not some form of organization, the data is effectively garbage. If you ripped out every individual page in the phonebook and scattered them all on the floor, it’s no longer an effective tool to find someone’s phone number; it’s just a mess of papers.
Data needs to be useful.
If we can’t do something with the data, what’s the point of having it? The temperature at the North Pole on January 1st 1989 is, by itself, inconsequential. But a history of temperatures at the same and similar locations, over a long period of time, gives us some great value — we can see trends, look for anomalies, and even predict the future of what those temperatures might be.
Databases need to be available.
Similarly, if we can’t access the data, what good is it? Databases are a technology, and like most technologies, they occasionally break. Again, most of that comes back to humans, because humans are the ones writing the code that creates the software that houses the data and runs the database, or that interacts with it. But of course we still have power failures, network losses, disk failures, and even solar flares. (Ask your favorite superstitious engineer; they’ll have at least one good story about a system outage that could only be blamed on solar flares or gremlins or the full moon.)
Databases need to be maintained.
Every DBMS has some kind of assumed ongoing maintenance requirements to keep it “running smoothly”. Just like your car needs an oil change every 3 to 8 thousand miles, your databases need periodic attention to retain all of those important qualities discussed above.
And the latest big topic, underscored by the GDPR:
Data needs to be governed.
This is a big topic for another conversation, but the gist of it is, data is generally “owned” by someone, and deciding who owns what, where it’s allowed to live, and how it’s allowed to be used, constitutes an entire sub-industry of rules, regulations, policies, tools, security practices, and consequences, much of which we’re only just beginning to shape and understand.
TL;DR: What do you actually do?
I currently work at a “small enterprise”, a business that has been around for some decades (as opposed to a Silicon Valley start-up who counts their anniversaries in months, like an infatuated teenager), managing their database systems. Some of that is financial/accounting, some is customer info, some is internal/operational, and all of it is important to at least one person in the business in their daily decision-making efforts.
Thus, I help ensure that the data is always ready, when it’s needed, in whatever form & shape it’s needed in. I model, massage, correct, enhance, and move it around. I help developers write faster queries (that’s a fancy word for “questions” that we ask of our data); I aide analysts with understanding and gleaning more value from the data; I maintain the underlying systems that house the databases and ensure that they perform well and get upgraded when necessary; and I work with business drivers (VP’s, CxO’s) to build reporting solutions that leverage the data to enable better, smarter decisions, and ultimately (hopefully!) increase profit. (This last part is actually crossing into the BI – Business Intelligence – job role, which tends to happen to most small-shop DBAs, because they’re usually in the best position to make that transition.)
If some of that sounds like a blurb from a résumé, it kinda is. This job has existed since the 80’s. But it’s always evolving, like the tech industry in general; so just because we’ve been around a while doesn’t mean we’re all old crusty bearded dudes. (Although we do have some prolific beards among us!)
So there you have it. Now you can tell your friends and family what a DBA does. Or at least, hopefully, I’ve helped my own friends & family understand a bit about what I do.
I loved my first job, no doubt. But slowly over the years, I started to feel pigeon-holed. I was always on-deck for problems that happened in the system due to what I’ll call “bad data”. If you’ve been in the industry, you know what that means – something in your datastore doesn’t “follow the rules”, or “breaks something that normally doesn’t break”, or “is just a little off-kilter”. (Otherwise known as “edge cases”, which I’ll talk about in a later post!) And often, it’s because the application allowed that data, which breaks the rules, into the system – Devs hate to hear this. But just as often, it’s because the data-guy, the DB-dev or analyst, ingested or transformed the data in a way that breaks the rules – so there’s plenty of blame to go around. And we knew that, and we respected that. We didn’t pass the blame, we just buckled down and tried to get it fixed.
But, too often, I was the one called upon to fix it, because I was the closest to the data – and yes, I knew it intimately, no doubt. Because of that, I started to feel left behind the curve. The other devs were pushing new tech stacks, learning new things, while I kept pluggin’ away at good ol’ TSQL (with the occasional smattering of C#). And yes, of course that’s partially my fault; I could have self-asserted, pushed for more learning opportunities, etc. I don’t regret my first job. I learned a ton – basically everything I know about databases, technology, and IT. But it was time for a change.
I started out as a data analyst, first playing in MS Access, and quickly moving to MS SQL Server. I also became a developer in the ASP.NET stack, way back in the 2.0 days (2006-07). Yet I found myself drawn to the data more than the code, for some reason. Perhaps it was the beautiful simplicity of the relational model. Or the myriad of variations that something as simple as an name field could contain & how challenging it could be to massage that “dirty data” into something elegant and consistent for the code to easily manage and present to the user. Or it could have been the thrill of analyzing a workload that may have taken several minutes to run, and then, by applying the right combination of indexes, relations, and set-logic, to bring that workload down to milliseconds.
This job was great. First “real” job out of college; a teeny-tiny software shop, 4 devs & the boss, working with K-12 school districts, making their lives just a bit easier when dealing with the vast swath of standardized testing and government-imposed student performance metrics that ultimately decided how they taught & how they got funded. And we grew, slowly but surely – 5 devs, then 6, then 8. We got to wear so many hats! Developer, Data Analyst, Helpdesk, Tech Support; even Printer, Delivery Crew, Trainer. It was truly a well-rounded experience.
As the years went by, it felt like family. And sure, every family has the occasional episode of dysfunction. But we were in it together, building up the business with passion, commitment, and respect. And those of us that were there since the start, we felt like we knew it all. Because we did; in that little microcosm, we truly were the experts. And each of us had a forte – I was the “data guy”, he was the “self-taught developer”, she was the “tech-evangelist / UX dev”, and he was the “dev architect”. Our best hire came soon after – the “young enthusiastic developer”, a true professional. Then there was our den-mother, bringer of snacks, paycheck-writer and book-keeper extraordinaire. And as always, the father-figure – the big kahuna, the boss man, who was more than willing to get his hands dirty and help with anything we asked him to.
That was a wonderful experience. In the near-decade I spent with that company and those colleagues, many of whom I consider my friends, I learned so much about technology, programming, databases, and business. And we welcomed more and more people into the family – the graphics & print-media design expert, the bright & cheery project manager, the ambitious salesperson, the kind & caring tier 1 support staff, the new data analyst, the veteran DB-dev, the expert architect, the UI/UX dev, the student dev, and even the Russian programmer! (They’re amazing, BTW.) Each & every one of them added value, and taught me something new.
I do not regret a single day of it. It was awesome.
But, as the baby bird has to eventually leave the nest, I needed to venture out, to get outside my comfort zone and see what else was out there, to join a new & different team and environment, to see if I was truly as good as I thought I was. Well, I think we all know the answer to that! (Hint: it’s “nope”.)
Actually, it’s been great – I drank from the proverbial fire-hose, learned a lot in a short time about a completely new line of business & a familiar but larger & much more complex tech stack & environment. So in general, I’m actually enjoying not being “the go-to guy” for every little problem that happens with the database.
And you know what else I discovered? I am pretty darn good at what I do. That’s a good feeling!
But I’m still the DBA. (Technically, one of two, but the other DBA is the company’s old-time-guru-turned-part-time-consultant). So in a sense, being that “go-to guy” hasn’t gone away, it’s just gotten much more manageable – there’s a larger organization of people around me that can help in many different ways. Whether it’s the old-timers with that tribal knowledge of all the hidden rules and nuances of the business, or the SysAdmin who can whip out a PowerShell script in less time than it takes to say “automation”, or the domain expert that can pinpoint exactly why something is broken because they literally “wrote the book” on that problem-space. So it’s really cool.
Plus, it gives me the space and the time to focus on getting really good at DBA stuff – performance tuning, report automation, data integrity, change management, backup/recovery, data security, all that jazz. I’m looking forward to really earning that “Senior DBA” badge – because even though that’s my title, I still recognize that I have a lot to learn. The landscape is constantly evolving – 5 years ago, this whole “Cloud” business was virtually (yes, that is a pun) unheard of; now it’s all anybody talks about. I “grew up” in an on-prem (that’s short for on-premises) Windows + SQL Server + IIS world; but now I’ve got my fingers on AWS instances (both EC2 and RDS), MongoDBs, a SSAS data warehouse, even a sprinkling of MySQL and Azure. Not to mention dozens of SQL Servers spread over 3 offices plus a data-center, and a ridiculously slick converged infrastructure platform called a “VxBlock”. The technology world just keeps getting bigger, better, faster, smarter.
Picture mid-2006. A fresh college graduate, BS in Computer Engineering. That’s half hardware, half software. Maybe more like 40-60. Not Comp Sci, though looking back, that might have been a little smarter. I learned a bunch about hardware and embedded systems that I will probably never use in my life. But hey, I can probably solder a capacitor to a circuit board without completely frying both it and my fingers!
So I had some education, but not much practical experience… none, really. But luckily enough, I found a job, in my home town, and started my journey! From data-analyst to programmer to database administrator and beyond, it was a wild ride. The technology space is a fascinating, painful, exciting, crushing, inspiring, challenging, awesome field to work in – and yes, those were oxymorons.
Anyway. Enough waxing dramatic for one post. Stay tuned for more about me! (Sure, it sounds self-centered, but isn’t that exactly what a blog is?)
But seriously. Thank you for reading and please do come back to visit. I promise I’ll attempt to entertain, inform, and intrigue. Or at least one of those.
In the beginning was the disk array, and all was empty and raw, and Windows Server moved over the face of the platters. And the DBA said: Let there be SQL Server. And there was SQL Server.
And the environment variables were set, and the disks were striped and mirrored, and the config was established, and behold spindle was rent asunder from spindle. And the DBA saw that all was in spec.
And it was day, and it was the evening of the first day.
And the DBA said: Let there be objects. And setup.exe brought forth myriad crawling things upon the face of the array. And instcat.sql brought forth all manner of tables and views that swim unseen beneath the waters. And procsyst.sql brought forth all the built-in procedures and all the operators of the air, that the users might be given wings and take fight over the data.
And it was day, and it was the evening of the second day.
And the DBA said: Let there be databases. And there were databases. And the system administrator looked upon the disk array and did see what the databases had wrought upon the disk arrays, and he did gnash his teeth and seek a new work upon the Internet with an engine of search.
And it was day, and it was the evening of the third day.
And the DBA created users. Male and female he created them. And he said unto the users: Thou mayest create tables and views as thou wilt. Yea, though mayest create even indexes upon the data. Only meddle not with the system database, for it is a holy place, and on the day wherein thou treadest upon it, on that day thy roles shall surely be revoked.
And the serpent crept among the users and whispered to them, saying: Thine roles shall not be revoked. Taste ye all of the system database, for ye shall know of b-trees and hints and ye shall be as DBAs. And the users heeded the serpent and did fill the system database with crap. And the instance did crash and the client did wax wroth at the DBA. And the DBA did gnash his teeth and partake of the fruit of the vine, for behold the users were permanent employees, and the DBA was but a contractor and could not revoke their roles.
And it was day, and it was the evening of the fourth day.
And the DBA did set default databases and default schemata, and did lock down all that was upon the face of the array with roles and encryptions and all manner of quotas, yea even from the transaction logs even unto the archived backup files.
And it was day, and it was the evening of the fifth day.
And the DBA created synonyms and links and did tune the server and apply patches upon the face of the server. And the DBA saw that is was good.
And it was day, and it was the evening of the sixth day.
And on the seventh day the DBA did rest from all the labors of the creation. But lo, his pager did ring and he ceased from resting, and did spend his sabbath on the phone with Microsoft support. And by the time the DBA got through to someone who knew whereof they spake, behold it was day, and it was morning of the eighth day.