Now you're thinking with Python...

Today I created something new and wonderful, a master database containing all the information on all the data Dr. Takamiya and I have collected for our research. This is a big database. We have 67,050 individual spectra which come in pairs (one red, one blue), for a total of 33,525 individual spaxels. Currently there are 46 different pieces of information for each spaxel, for a combined total of 1,542,150 pieces of information contained therein. The information for each spaxel was spread out over 5 different files prior being collected and organized (that's 5 files for each spaxel, there are actually a total of 596 files for the individual spaxels, plus one more that contains certain information for each group of 225 spaxels that makes up one-half of an observation). This is a single text file that's 11 megabytes in size, bigger than the pictures my camera takes at maximum resolution (though not so large as to be unmanageable).

I'm pretty excited about this as you can imagine. This is the culmination of over a year of work on my part. I've really enjoyed working with my mentor Dr. Takamiya for the entire time. She wonderfully handled the balance between letting me try and fail and learn on my own and being there to help when I really couldn't figure something out. For the first several months I was too ashamed to admit that I didn't fully comprehend the big picture of what we were doing, but she was always willing to explain it to me again and able to show me the next little step to take to get there (and yes, I finally understand it now). When I say the database is the culmination of work on my part, I really mean it; while she showed me the way to go, she left nearly all the actual design and implementation decisions up to me, and the process indelibly bears my mark in many ways (for good or bad). I've grown in so many ways as a Python programmer during this time I can hardly begin to list them all, though I think the biggest is my recent sudden understanding of Python classes, something that would never have happened if I hadn't spent so much time thinking and struggling and working with Python over the last year.

Of course, this reminiscing isn't to say that we're done. Far from it! In some ways the real work is just getting started. Now we can begin rigorously plotting the data to really tease out the relationships that have been hinted at previously. That should keep us occupied for some time...