If you’re like many IT professionals who’ve had anything to do with large amounts of data, you’ve become immune to the phrase ‘big data’. Mostly because the meaning behind that phrase can vary so wildly.
Processing ‘big data’ can seem out of reach for many organizations. Either because of the costs in infrastructure required to establish a foothold on this front or because of a lack organizational expertise. And since the meaning of ‘big data’ can vary so much, you may find that you’re doing ‘big data’ work and then ask yourself, “Is this big data?” Or an observer can suggest that something is ‘big data’ when you know full well that it isn’t.
With my own background in data, I’m ever curious about what’s out there that can make the threshold into ‘big data’ seem less insurmountable. Also, I’m interested in the security considerations around these solutions.
In the last week or so, I’ve gotten more familiar with AWS s3 buckets and a querying service called Amazon Athena. Here’s the truly amazing thing. You can simply drop files in an s3 bucket and query them straight from Amazon Athena. (There are just a couple steps to go through, but they are mostly trivial.) And for the most part, there’s not much of a limit for how much data you can query and analyze. You can scan 1tb of data for $5. What? That’s right. And you didn’t have to set up servers, database platforms, or any of that. I’ll be exploring Amazon Athena more and more over the coming weeks. If you have an interest in this sort of thing, I suggest you do the same.
One note: Google has something similar called BigQuery, so that might be worth a look as well. I’ve explored BigQuery briefly but I keep coming back to various AWS services since they seem to be holding strong as a dominant leader in emerging cloud technologies. But as well all know, the emerging technology landscape can change very quickly!
For some time, I’ve been interested in learning about the Raspberry Pi. It’s little a bare bones computer that packs a big punch. And to top it off, it’s quite affordable. Through work I heard about a way to use a Raspberry Pi for an OS called Retropie. Retropie is an emulation platform that let’s you play scores of old games…if you have the digital files for them, of which many can be found with the help of Google.
I’m not much into modern video games, (as in games from the last 20 years or so), but I did play NES games back when I was in jr. high and high school. And I do still have my original NES, but it has a number of issues that make it less than reliable for playing. My kids are interested in the older games because I’ll actually join them when they play. And, quite frankly, because the older games are super fun to play and easy to learn.
Anyway, Retropie is a great way to learn how to use and get familiar with the Raspberry Pi. You simply, burn the Retropie image on a micro SD card, pop it in the micro SD card slot and boot it up! There are a few other things you need to know, but that’s the gist of it. Get a few games, a controller or two, have a monitor with an HDMI plug-in handy and you’re good to go. That’s a bit of an over-simplification, but please do explore Retropie and Raspberry Pi if you’re at all interested in this sort of thing and are looking for a good way to get familiar with the Raspberry Pi world.
Here are a couple key links:
These days efforts to revamp company culture are in vogue. I’m going to attempt to articulate what I see as a connection between machine learning and efforts to change company culture. Stay with me here a bit because the analogy doesn’t show up until the fourth paragraph and I need to share a little bit of background first. 🙂
One group leading the charge to change company culture is Partners in Leadership (https://www.partnersinleadership.com). They use a tool that identifies the following flow toward changing results. It’s a pyramid that moves from experiences to results in the following steps: EXPERIENCES >> BELIEFS >> ACTIONS >> RESULTS. According to the model, you start with the results you want to see as an organization and then move backward until you’ve arrived at the experiences that you need to create. The thinking is that experiences shape beliefs, which shape actions, which shape results. They maintain that you cannot simply skip ahead results until the rest of the house is in order first.
As for the experiences, they actually need to be high quality experiences. Partners in Leadership breaks these experiences into four types (big paraphrase here): 1) Easy to interpret, 2) Needing work to interpret, 3) Very little meaning, so there isn’t much to interpret, and 4) Experiences that, well, kind of did the opposite of what they were intended to do.
Now it is time for the machine learning analogy! Boiled down, machine learning is essentially learning from experiences (data) in order to shape beliefs (trained statistical models). These beliefs/models turn into actions (acting on the outcome of a model), which leads to results. Critical to this process is the experiential data and its interpretation (the model). We train our models by feeding data (experiences) into them. Why am I making this connection? Because organizations are really struggling to understand machine learning. Why not piggy back off of something that they’re learning already? Results from machine learning algorithms are no different results gleaned from an organizations’ cultural change initiatives. What data do you have that you can use to shape your statistical models? Which actions do you need to take to get results? You can change your culture and understand machine learning at the same time!
I spend approximately 8-10 hours a day in front of a computer. That’s a lot of time staring at a screen. (I think a lot of other people are probably in the same boat.) And, yes, I’m sitting in front of a screen to write this. 🙂
So I’m mindful of ways where I can dive deeply into the analog world. I’ve found one activity really provides a great escape from all of that: analog music. Yup, an actual musical instrument. Lately, I’ve been playing the violin. It is so incredibly fun and there is so much to learn about it. Granted, if I want a tip from Itzhak Perlman about how to hold my bow, I briefly turn to YouTube for a quick tutorial, but then I’m right back to my purely analog endeavor. I also play guitar, cello and mandolin. All those instruments provide an excellent balance against computing.
For me, the vibration of an actual string, which is caused by fingers, hands and arms…and then the resulting sound dancing off my eardrums…is about as real as it gets. Sure, I can have my head in some sheet music, but I can also close my eyes and visualize the sound and have it connect with actual movements my body is making.
Also, I try to enjoy every note and try not to get to wrapped up in a whole piece or song being completed. Sometimes three notes are all you need, or a couple measures. Just ask the members of my household. I’m sure there are times when they wish I had a slightly more varied approach to my practicing. In my mind, though, practicing by definition is repetitive. Anyway, something to think about as an antidote to computing. Never too late to start!
How much of the world’s IT infrastructure is in the cloud now and much of it will be in the cloud in five years? I’m sure there is nearly solid data somewhere to answer those questions. Regardless, it is happening and it won’t be long until most IT infrastructure is in the cloud.
Oddly, though, in my conversations with other IT professionals, it seems like we’re finding we’ve arrived late to the party. With the advent of “the cloud” organizations are finding that there are all sorts of solutions out there that don’t necessarily need the involvement of traditional IT. In much of the IT world, our perception is that this process is more gradual when in fact it is accelerating.
So the real question is not whether “the cloud” is coming, but whether we see it coming. If we want to make sure cloud implementation is done properly and doesn’t completely hose our respective organizations, we must learn as much as we can in a very short period of time.
Nearly every day I find myself reading about cloud security risks right along side incredible cloud solutions for problems that would normally be much harder to solve. At the same time, many cloud solutions create problems that we’ve never seen before. With the flip of a switch something private can become public: see S3 buckets. And it isn’t so much that the cloud is insecure, but how we connect to the cloud, whether this is through our API infrastructure or open ports that maybe shouldn’t be…open. The only answer I have for all of this is that we need to learn, learn, learn, learn…and fast.
So, generally, the easiest way for hackers to get into an organization is by convincing users do to something: click on an email attachment or a link, make a phone call, share information, etc. For all the technological advances that have sprung forth in the past decade, this is still among greatest challenges faced by security professionals: figuring out how to keep people from following hackers’ instructions.
Our biggest vulnerability is also our greatest asset. We can make thoughtful decisions quickly. And sometimes our decisions aren’t so thoughtful because we’re in the midst of doing other things, or generally too distracted to slow down and think through what is being asked of us. This little glitch in our code is all an attacker needs.
Exploiting this human vulnerability is all an attacker needs to get us to act in a way that is not in our best interest. This is the nature of a hacker-victim relationship. But are there other ways that people are getting hacked that maybe aren’t as overt as this? Think of the decisions we make daily. How many of them are in our best interest or the best interest of our friends and family.
We make snap decisions all the time that aren’t really based on sound logic. I bet any one of us can look back over the course of the case and think about an action we took that wasn’t ideal. It’s a given. If we didn’t make decisions relatively quickly, our brains would grind to a halt and we’d become mostly ineffective at making our way through this world. But as technology gets better and better at humans hacking other humans (think targeted advertising through machine learning algorithms), we should pause to ask ourselves whether we’re on the right track. Will this lead us to a better humanity? Just throwing that question out there. It can go a myriad of different ways. Thanks for reading.
I’m one of the last remaining people in this world who don’t have a smart phone. I’ll often be in a group if fellow IT professionals and pull out my flip phone to check the time. “Is that a flip phone?” someone will ask as they lean forward in their chair and peer down at my hands, attempting to figure out what would cause someone who is steeped in technology every day to carry this sort of relic. As I lift it up, all heads turn my way, mouths open and nostalgic signs fill the air. Onlookers talk of a simpler time when we were free from 24/7 social media and and subservient to on-the-fly navigation from real maps and sheets of MapQuest printed haphazardly on white glossy paper, barely legible.
Because I don’t have a smart phone, I’m often looking for alternative ways of doing things that people normally do with smart phones. One of these things is MFA or multi-factor authentication. If you use AWS, it is generally a good idea to have MFA for your root account. They don’t allow SMS for their second factor any more, so I spent a few hours looking around for an option. I did look at some hardware keys like the Yubikey, which I may check out eventually, but I needed something cheap and now. Enter the “Authenticator Extension” (https://github.com/Authenticator-Extension), which you can get through your Chrome browser extensions.
It is TOTP compatible, which covers quite a few sites. I can use it for AWS, Facebook, etc. I probably won’t use it for LastPass because I’d like to have a paper backup of such a critical second factor. The “Authenticator Extension”, as it is so generically called, works great for me using my Chromebook, which often acts as more cumbersome, less convenient, and less connected Smart Phone for me. I recommend it.
I’ve been a Linux user at home for quite some time. We were a Windows family very early on but ran into issues with viruses. I resurrected a super old laptop and put Lubuntu on it and gave it to my wife. It worked well for years. After a while, one thing or another wouldn’t work, so on a whim I got her a Chromebook. Nearly everything she does is online, and she’d already started using Google docs when on the Lubuntu PC. As a result, the transition was peachy! After watching her tote that thing around the house for a year or so, and noticing how carelessly she worried about charging the battery or booting it up, I decided I needed one too!
It’s done quite well for me. Occasionally, I have to jump over to my Ubuntu desktop for more high-powered activity, but 80% of my computing at home is on the Chromebook. This experience and the evolution of computing as it moves into the cloud is leading me to believe that the days of everyone running around with what is essentially their own personal server, are numbered. I’m guessing in about five to eight years, computing will be cloud focused even more than it is now and people won’t really own traditional laptops any more.
I’ve got just about everything marked off my list on the AWS learning front for the weekend. This domain is now transferred over from my old host: jeshuaerickson.com. I started up a WordPress instance using AWS Lightsail. Then I assigned a static IP to that instance. I also got my DNS zone set up. Finally, I got my SSL cert completed and integrated with Apache. (There are pretty straightforward Bitnami guides for this. It’s not done through the regular “Services” interface. Just remember when you’re in Lightsail, you’re in the Bitnami world now!)
The other piece that I worked out was taking a snapshot and doing a restore, which is basically getting rid of the old instance and assigning the new instance to the static IP I created. I was expecting to see a “restore” button in Lightsail, but that’s not how it works. Makes sense now that I’ve gone through the process once. (I had to do this because I hosted the SSL cert integration the first time around.)
Throughout all of this I am attempting to keep track of AWS billing. I had some Directory Service charge pop up and didn’t find it until I started poking around in another region. I did some quick back and forth with AWS and got a credit for those charges. Ultimately, I was very happy with how responsive they were. The variety of services that they offer inside AWS is INSANE and billing can get a little tricky to navigate if you’re not familiar with the AWS administrative console.
“Sunday Solace” is a Spotify playlist I created. I listen to it on Sunday nights (and, well, other times, too). It gives me solace.