es Heavy Industries

My Adventures in Self-hosting Part 2

esell@esheavyindustries.com (esells) — Mon, 24 Sep 2018 13:10:30 -0600

Over a year ago I published a post on my initial adventures in self-hosting outlining what I was using, what I was trying to get rid of, etc. Awhile back a friendly e-mail came in from someone who had read that post (thanks!) suggesting I follow-up with how things are going and any other tools I started using. Well my friend, here is that post!

Let’s start with what has been a success…

Creating and hosting this site with Hugo, Caddy and Let’s Encrypt has been fucking awesome. I don’t use any complicated build systems like I’ve seen others using. For me I am happy with a git repo where the output from Hugo is stored and then the Caddy git plugin. Write content, use Hugo to generate the site and then do a git push and a minute or two later the new post is live. Doesn’t get much easier than that. ¹⁰⁄₁₀, would buy again.

While I’m talking about this site, using GoAccess has been great as well. For me (again), it works fine and gives me the data I care about. Does it have all the functionality of Google Analytics? Nope, but I don’t need or want that so I am not missing anything. Another great example of software that just works. My only minor complaint is the current lack of ability to filter dates/times in the web UI to look at a specific range. Depending on how your logs are setup though this might not be a big deal to you. I’ve also seen a pretty easy way to mimic this functionality but have not tried it yet.

Password management was one of the larger things I wanted to get up and running and had two great experiences. In my previous entry I talked about pass which is a fantastic tool. Unfortunately, it doesn’t work well for groups (a family in my case) and non-technical users. I needed a way for my family and I to store and access various logins on our mobile devices and our laptops so I decided to take a look at Bitwarden. After a day or two of usage I was confident that this was the solution for us. The “install” is painless as the suggested method is via Docker. The requirement of SQL server is kind of a bummer, but again, containers. If you really hate SQL server, there are many 1:1 compatible backends written in various other languages that don’t have this requirement. How about the clients? Fantastic. For an open source project, the mobile clients are some of the most polished I’ve seen and easy-to-use for the non-technology folks.

It wasn’t called out in my first post, but I also ended up running my own Wallabag instance and it’s awesome. Not really much to say outside of the fact that it works and gives me a way to keep track of all the articles I’ll never read. Go give it a try.

Now for the “failures”. I have failures in quotes because while some of these ended up not working for me, it typically had nothing to do with the product itself, but the ability to get others on board.

First up was my self-hosted code mess. If you look at my first post you’ll see that I had decided to use Gitea as well as Drone and a bit of custom glue and magic. This whole setup was great and I was really happy with it, the problem is that it made it really hard to get other people to use it. I kept mirrors of my repos on Github with a note pointing people to my self-hosted site thinking that would help a bit, but it didn’t. The first problem was that in order for someone to open an issue or PR on my self-hosted repo they needed to have an account. Gitea added the ability to auth with your Github account which was nice, but that didn’t totally solve the problem. As an example, take a PR. On Github you fork the repo, make your changes and use the fancy PR button to create the PR at the upstream repo. Well, since nobody else used my self-hosted Gitea instance, nobody had code there. They were typically (100%) forking from Github which meant you couldn’t create a PR with those changes on my self-hosted Gitea instance. The solution was for them to come over to my self-hosted instance, login with their Github credentials, fork the repo on my instance, make changes and then create the PR. Not impossible, but kind of painful and doesn’t help people contribute to my small amount of shitty projects.

I dealt with this for a year but have just recently decided that it just isn’t going to work. The whole concept of Github is to create a social site where folks can contribute code. I like that. I like helping out when I can and I like people sharing cool ideas with me on my code. This wasn’t happening for me and my projects are nowhere near popular enough to force people over to my self-hosted instance. For those reasons I’m actually going to move back to Github as my primary code repo and just use Gitea as kind of a mirror. I realize that is a little silly with Git but fuck it, Gitea is already up and running so I might as well use it for something eh?

E-mail was something I touched on previously and as of today, I don’t have an answer outside of 3rd party providers. This was probably my 3rd attempt at hosting my own mail server and sadly, again, it didn’t work out. The issue was deliverability and no matter what I tried I couldn’t get it to a point where I felt that my e-mails would reach the recipient(s). Sending to the major e-mail providers was a game of roulette on if my messages would be flagged as spam despite the fact that I had done everything that they all suggest to do.

Clean IP? Check.

DKIM setup? Check.

SPF? Check.

I REALLY wanted this to work but it just wasn’t meant to be. I’ve decided to just stick with ProtonMail for now :(.

Photo and video sharing was also something I had not setup yet in my previous post but something I was looking into. I ended up setting up Seafile which as a product worked well. I still have some odd issues with the Android client syncing my photos, but I know about it and can use my manual workaround to “fix” it. The bigger problem was the ability to smoothly play videos which doesn’t really land on Seafile. Most of my family lives far away from me so videos are something I take a lot of so that the family can see what is going on in our life. With just a single VM, no CDN and a small internet pipe it was really hard for people to watch the videos because they were constantly buffering. Technically the video part was working, it just wasn’t very usable because of the starts and stops. This was a huge bummer because Seafile itself is pretty cool. For now I’m using it as another spot to backup all of my data just in case I need to split from Google Photos quickly, but for day-to-day sharing I am still using Google Photos :(.

Why Go?

esell@esheavyindustries.com (esells) — Thu, 30 Aug 2018 13:19:13 -0600

Background

This week I am lucky enough to be attending GopherCon which is a great conference that is focused on topics that revolve around the Go programming language. As I was sitting in one of the talks on day one something hit me: I don’t really belong here. This had nothing to do with gender, age, hair color etc but more to do with the fact that I was listening to seasoned developers talking to other developers about hard developer problems. It wasn’t necessarily the content that triggered this thought, it was more the fact that I don’t write code everyday and when I do, it is typically for side projects or for stuff my job doesn’t pay me to do. I am not a developer, I am a nerd who likes to play developer a handful of times each month. So if I’m not a developer, why would I spend the few “coding” cycles I have to write code using Go? Why not Node.js? Why not Python? Why not ____? Well, let me tell you why…

Go is simple, and I mean that in the best possible way.

But wait, let’s back up a bit…

My entire career has been in technology. I’ve done helpdesk roles, sysadmin roles, architect roles etc. Shit, I mean at one point in my life I was a tester for Palm Pilots. Along the way I’ve always played around with code using languages that anyone over 35 probably has touched. Early on it was Perl and shitty shell scripts. Later on I graduated to PHP. I played around with Ruby and Python and of course, tried my hand at C (doesn’t everyone?). Most of those I used at a specific job but then ditched them once I moved on. Then came Java. Oh Java and your incredible mess of libraries and heavyweight IDEs. I actually really liked Java and ended up using it for the one program that a company I worked for ended up using long-term. As a sidenote, I feel bad for the person that inherited that mess. Anyways, the point of all of this is to show that I had played around with many languages so, even though I am not a dev, I have seen enough to know what works for me and what doesn’t.

I was just coming off of my Java kick and was not feeling super excited about the Java ecosystem. It felt like anytime I wanted to write an app I needed to import 25 libraries, which then needed to import 50 other libraries. Then I needed to dick around with XML files and all sorts of other annoying shit to finally get an app that was painful to package. Not being a real developer it might have just been that I was doing it wrong, I’m not sure, but while writing the Java code was fun, everything else sucked. Then, on one summers eve, I saw a post on Hacker News talking about Go. Being a nerd I decided to give it a go (ha!) and check it out. After about 30-40 minutes I was all in, this was what I had been looking for.

So Why Go?

Strap in, I’m about to go wall-of-text on you…

What was it about Go then? I have long subscribed to the KISS principle, so when I started playing around with Go I fell head over heels for it. I hear people call Go boring and sometimes a bit verbose, for me that was exactly what I liked about it. It was simple in the sense that things were much more readable to me. I’m not talking simple as in you can start being ten times more productive in Go because it’s really easy to learn or something, I mean simple in the sense that the code you write ends up being self-documenting, even for non-developers. Not being a dev it was awesome to be able to understand everything that was going on in the code. I’ll use error handling as an example. People bitch about the error handling but I love it. I can see, without any magic, what is happening and how it is being handled. To this day I still do the tried and true

    if err != nil {
            dostuff()
    }

because it is 100% obvious what is going on there. Sure it takes a bit more time to type out but that doesn’t bother me. The lack of magic in writing your code is balanced with some magic that someone like me wants such as memory handling. That is the kind of magic I would rather not know about so have at it Go!

This has been the biggest reason why I’ve stuck with Go. I don’t have to worry about models, objects, build tools (Maven, I’m looking at you) or any of that shit. Could I implement my code that way? Probably, but I want my code to be as easy to read as possible. I usually have 2-3 files(remember, I am not a dev) in a single, flat directory. I can see you freaking out right now, but this works well for my small projects. I am able to separate the code so that specific things share the same spot/file, but I don’t need to dig through three directory levels only to find 10 files, one of which contains what I actually need. Yes, I know an IDE can help with this but that is another topic and I’ll touch on it later.

What about libraries? Yeah, they exist (and can be super helpful) but 90% of the code I write never needs them. Writing a web app in Go compared to something like Java is about as opposite as you can get. Web frameworks? What? My web framework is the Go standard library + the awesome HTTP Router library from Julien Schmidt. I get just the right amount of control all while reducing some of the more tedious stuff. Oh, and that library is like ~1k lines of code so if I do need to figure out what it is doing, it is pretty easy to poke around and find what I need. Did I mention the Go standard library? That thing is amazing, and as a cherry on top, it is well documented. When I am writing code I typically have two things open: my editor and https://golang.org/pkg/. In fact, probably half of my code is more-or-less cut & paste from the documentation. Seriously.

Guess what else I like about Go… No need for a heavy ass IDE. I can write code in Vim all day long without any issues. There are helpful plugins like vim-go, but to be totally honest, you don’t really need them. I do use vim-go but only about 20% of it. I don’t do code completion and I don’t use any of the special bindings or commands. I do like the automatic gofmt and goimports on save though :). Working at Microsoft I feel like I really should be using VS Code but I haven’t been able to pry the Vim safety blanket from my hands yet. The point though is that you could write Go code in notepad if you wanted to. Yes, I realize you can do this with many other languages as well so you don’t have to send me angry e-mails.

Are you bored reading about my love for Go yet? No? What if I told you that you could compile your code to a static binary that anyone could use without any extra shit to install? What if I told you that you could cross-compile your statically linked binary for most of the major operating systems out there without much pain? I really like this part because it makes things so easy to share your app with others. I can write my code on OSX, cross-compile it for my Linux servers and then scp it up there and it just runs. I don’t have to install anything extra, it just works. This of course assumes you are not using import "C" in your code. All hell breaks loose when you do that. Luckily those cases are few and far between for me.

Are you ready for your head to explode? I actually am ok with the fact that Go doesn’t support generics. To me this goes back to my first point about things being readable and easy to understand. I have run into a few cases where the lack of generics was a bit of an annoyance, but I’ve always been able to work around it with a bit of extra code that ended up making everything easier to understand (in my opinion).

Every other “Why I Love Go” post talks about concurrency, but I haven’t really needed to use Go routines too much in my work so I won’t pretend that I have experience with them and ramble on about how awesome they are. Everyone else says they are awesome so I’ll trust them.

Go also has the best mascot of any programming language out there.

Is Go Perfect Then?

Of course not! There have been times that Go has pissed me off, many of those things are well publicized by other Go users (and non-users).

Versioning being in limbo for awhile has hurt. I’ve gone from no versioning to GB to Dep and am about to start kicking the tires on modules/vgo. This has never held me back with regards to writing code, mainly because I try my best to stick to just using the standard library. When I have done versioning of my code I’ve just put a note in the README about what tool the versioning was done with and how to use it. A minor annoyance but nothing I’m going to lose sleep over. Locking this down with what appears will be vgo would be really nice though and likely inspire others to start versioning their dependencies more often.

The binary that Go spits out after it compiles your code can be a bit hefty as well. This isn’t much of a concern these days, but I grew up in the megabytes of memory days so I still tend to get a bit touchy about file size. Having the binary statically linked drastically outweighs the size of the file so I’ll take that trade-off all day long.

A lot of people complain about the whole GOPATH thing, but I honestly don’t care. Again (picking up on the theme here?), I am not a developer and I don’t work with other developers on a team, so having things forced on me like that isn’t a big deal. I’ve yet to run into a situation where I was annoyed by where I had to place my code on my filesystem. I do believe that changes to that requirement are in the works though(?).

Really that is about it. This is not meant to be a “Go is the best ever, you need to use it NOW” type post, it is simply a post on my experiences using Go. My hope is that some of my non-developer friends will read it and give it a try. If it works for you then that’s awesome, if it doesn’t that’s ok too. It is all about using the tool that you feel comfortable with. If that tool isn’t Go we can still be friends, we just won’t be able to use the secret gopher handshake ;).

My Adventures in Self-hosting

esell@esheavyindustries.com (esells) — Fri, 03 Mar 2017 21:10:30 -0600

Some people’s new years resolution was to lose weight, others was to give themselves more “me” time. Mine was to move off of as much SaaS offerings as I could in an attempt to start taking back my privacy.

Don’t get me wrong, there are a lot of SaaS solutions out there that are awesome and take away the burden of having to manage a server but I felt that there was just too much of my info being spread across these sites. There wasn’t a single event that triggered my new goal to move everything to my own infrastructure, it was more that I was noticing that every couple of months I felt like I was signing up for new SaaS apps or connecting to a new app via another service I already had an account on. I had already closed down my social media accounts, which were probably the ones I had to worry about the most, but wanted to take the next step to independence.

First on my list was this site. I had been thinking of moving it for some time as hosting it on Github pages didn’t allow me to use SSL unless I used something like Cloudflare which I didn’t want to do. Most of the big cloud providers have some type of blob storage that works great for static sites, but again, at the time none of them had what I would call acceptable SSL support. The solution I ended up going with was a small VM with one of the large cloud providers and setup disk encryption (with my key) to make sure it really was “mine”. Now I needed a web server. I had been an Apache guy most of my life but that felt too heavy for what I wanted. I had heard of Caddy and after reading the docs it sounded like a perfect fit. It was easy to setup, has support for Let’s Encrypt out of the box and also has a nifty git plugin that pulls in my static site from my git repo whenever the content has changed. It took me about 5 minutes to get things up and running at which point I had one item off of my checklist. Caddy is awesome and as you’ll see later, it ended up being the hub of most of my solutions.

The next item was how to get off of Github and the random services I had connected into it like TravisCI and Coveralls. The part I struggled with here was the social aspect of it. Not so much stars, watchers, etc but more of the PR and issue management system. Granted I don’t have any high-profile repos but at the same time I didn’t want to put up a barrier to prevent people from submitting something. Asking someone to create a user/pass just for my shitty git repo wasn’t an option. As luck would have it, Gitea, which is a fork of Gogs had just sprung up. Gitea has been much more active with regards to commits and one of those commits just happened to be supporting authentication via Github! I pulled down the source, got it all built and dropped it behind my Caddy server, again with it’s own fancy Let’s Encrypt cert. Is Gitea a bit rough around the edges? Sure, but it covers everything I need and is fairly lightweight, easy to setup/maintain and can use sqlite as a backend so I didn’t have to manage a database server.

So I had a solution to my git repo hosting, now what about replacing TravisCI and Coveralls? I’m glad you asked! This part of my self-hosting process ended up being a bit more work than I expected. To replace TravisCI I went with self-hosted Drone. Drone was pretty easy to get going since it comes as a Docker image so once I had it fired up I simply logged in and enabled it for the repos I wanted. The .drone.yml file took a bit of work to get working with the native go vendoring system but I eventually got it working. Coveralls was next up on the list and surprisingly I didn’t find very many open source options. I ended up hacking together hoptocopter which actually just uses the native go test coverage tooling and then spits out one of those fancy badges via shields. To avoid having to call their API I used the wonderful shields docker image provided by beevelop. From there I just setup a Docker compose file that started up hoptocopter and shields together and bam! Instant code coverage badges. At this point I had basically re-created everything I had in Github. There were some things I couldn’t implement such as builds being triggered by PRs, but that wasn’t a huge deal to me so I considered it a success. Guess what I used to front these? If you said Caddy you’d be correct.

Previously this site had used Google Analytics for metrics. I actually like the product and enjoy seeing all of the cool data about my site but Google and privacy do not belong in the same sentence. My next goal was to rid myself of Google Analytics and find something I could run on my server but that still had an easy to use UI. Enter GoAccess. Really nothing to say about GoAccess outside of it just works. It parses the log files from Caddy and turns them into a pretty html page. This page is served up by… Caddy.

Password management, a great debate topic. Do you use a fancy hosted tool like LastPass or do you use something you can store locally like KeePass? The answer for me was neither, I went with pass which is a super simple (good) way to manage your passwords but still be able to access them on most devices including my Android phone. I setup pass to use git as a storage location by setting up a private repo on my new Gitea setup. I lost a tiny bit of functionality over something like LastPass, but nothing I couldn’t do without. If you wanted to be super crazy you could use something like git-remote-gcrypt to encrypt your actual git repo. Double the encryption, double the fun.

Now I have to talk about the things I haven’t come up with a good solution for (yet). The biggest is e-mail. A few years ago I tried hosting my own mail server and did everything the internet says to do to avoid your e-mails being flagged as spam. Unfortunately that didn’t work and about 80% of my e-mails ended up in the spam folder of my friends and family. I am currently using Protonmail which is great and I have no real problems with, it’s just that I don’t have those messages on my own server so I have to trust that everything they say they do is actually true. That’s a fairly big thing to do but at this point I just have to suck it up and deal with it. The second area is image and video sharing. Things like Google Drive or Google Photos are great products but again, Google. I’ve been looking around for some open source alternatives and haven’t found anything yet that meets my needs. I want to be able to send photos right from my phone to the app on my server and then send out generated links to family members to access those files all while keeping them hidden from everyone else. I’ve toyed around with hacking something together but I have to believe that an app like this already exists so I’ll keep searching.

I still have a bit left to do like moving all of my Github repos over to my Gitea setup but so far I’ve been enjoying the new freedom. Since most of these apps are pretty low maintenance I don’t really have to do much sysadmin type work which is great. If that will still be the same six months from now I’ll just have to wait and see.

I owe my career to being a script kiddie

esell@esheavyindustries.com (esells) — Fri, 13 Jan 2017 21:10:30 -0600

It’s an uncomfortable feeling when you have to tell your mom that the internet service she had been paying for had just been terminated.

For the second time.

Or at least it was for me.

It was the mid to late ‘90s and the latest craze was the statd exploit that had been released for Solaris. I had a list of potential targets (thank you nmap) and wasn’t really looking to do much, but being young and stupid I thought I’d give one target a quick test without using any of hiding techniques that typically accompanied a more thought out attack. Fifteen minutes later I was explaining to my mom why she wouldn’t be able to read her e-mail anymore. She didn’t really mind as she wasn’t a big fan of Pine.

So, let’s go back a few years… Your typical upper lower/lower middle class home. Single mom who is dating a dick. Young male teenager. The situation made-for-tv movies are usually based around. The internet has just started creeping into the mainstream and AOL is king. Windows 95 is the shit. My mom and her boyfriend had just come back from some create your own business rip-off conference thing and were convinced they were about to make it big reselling random shit. In order to be a legit business though they needed a computer, every business had a computer right? At that point I was the happiest kid around because my school had started accepting typed reports instead of handwritten and my family was about to get a computer. I could have given two fucks about the internet, I just wanted a text editor and a printer. It took a while for them to build up the computer fund but they did it and I can still remember going to K-mart to get some generic, shitty ass pentium 133Mhz PC. Of course the great business plan failed and the computer ended up being primarily used by me. Early on I just did school reports but at some point the AOL bug bit and I found myself doing what most teenagers on AOL did at that time: acting like an idiot in chat rooms. Now I had ventured out on to that scary world wide web place but not very often, everything I needed was right in AOL and life was good.

Then my life changed.

A person in a random chat room was “hacking us”. When I say “hacking us”, I mean spamming the shit out of the channel to the point where nobody else could get a word in. I had heard of “hacking” before, my friend and I had even created an encrypted file once on an old Apple with the flying toasters screen saver, but this was the first time I’d seen something like this. How could this single person essentially take over and control a chat room managed/owned by AOL? I had to know how this person was doing this and so began my journey…

We’ll skip over the boring part where I started creating websites with flaming skulls, loading up the latest AOL scripts and playing around with virus “generators”. The important thing that happened during this period was my discovery of texts and zines.

As I started to go deeper down the rabbit hole the need to read everything out there started taking over. There were times where I wouldn’t go to bed but instead would lock myself away in my room, read texts all night and then go to school in the morning with little to no sleep. This new world I was discovering was amazing, you could cobble together some nonsense (code) and end up with something that allowed you to gain access to another computer! You could take a fucking whistle from a cereal box and use it to work your way through phone systems! Sadly, by the time I had come across most of this stuff it had already been fixed but that didn’t matter, the fire had been lit and now I was all in.

All the “legit” people were using Linux so I spent a week downloading the different packages for a Slackware install and put them all on 3.5” floppies only to have the install fail. I should mention that up until this point I had basically zero Linux experience. Luckily for me there was Cheap Bytes which was a site that would burn everything to a CD and mail it to you for a small fee. A few weeks later that old 133Mhz Windows 95 computer was a lean and mean Linux box. I can still remember the panic when I saw that “darkstar login” prompt come up. What the hell had I done? As I said earlier, I was all in.

For a while I just spent my time learning what the hell I was doing. Things that seem basic today were quite a bit more interesting back then. An example would be setting up your PPP dial-up connection when you had no clue what PPP was. These first couple of months with Linux were really the first time in my short computer using history that I had to start problem solving and reading documentation to figure out what to do next. A lot of it didn’t make sense but I was able to piece enough of it together to get a functioning system up and running that would allow me to connect to the internet, send and receive e-mail and browse the web with lynx. The satisfaction of building my own Linux kernel was pretty awesome. Sure I was just following a menu-based system, but the fact that I had to take source code and turn it into this working blob of magic was pretty badass.

At this point I was in full on consume mode. My nights and early mornings were still being spent searching and reading. Places like linenoise.org, bronc buster’s forum, the l0pht and technotronic were like fucking crack. I had the hacker’s manifesto memorized. I knew all about war dialing and to prove it I had one angry fellow call my house and leave a message after my war dialer had called him multiple times between 1-4am. Luckily I was able to delete that message before my mom heard it. My friends were playing video games, watching TV or doing homework at night while I was reading monospaced font filled text files one after another, writing perl scripts and learning the amazing world of awk and sed.

Again we’ll fast forward a bit to a time when the ashtray lumberjacks, ytcracker and milw0rm were making waves. I was in a small but tight knit group (telehack.net yo!) whose main fame, aside from a few webpage defacement shout outs, was our large stash of texts and zines. Also that time we had a “feud” with the UPL. We were all addicted to learning so we just took all of the files we already had locally and combined them into a single site. Outside of that we did what I think the majority of bored teens in our world did back then which was hang out on IRC 80% of the time, dabble in some gray area security things and read. I was lucky enough to live in an area that had a fairly vibrant hacker culture (_THE_ 303) and although I never made it to a 2600 meeting on the 16th st mall, I did meet a few like-minded individuals to help bolster my skills and add to the knowledge I was accumulating.

So here is where I talk about how I never liked school but was still able to get amazing grades, go to MIT and become some badass computer/technology guy. Unfortunately (maybe fortunately?) that wasn’t me. I didn’t like school but because of that I never went which meant I got shitty grades. I didn’t go to college because I had shitty grades and was never going to be able to afford it without some type of scholarship. I’m not a genius or one of those people who think at another level. I’m just a guy who likes to learn and tinker.

What does an 18 year old who isn’t going to college do? Get a job so they can GTFO! A few weeks after I had graduated high school I was interviewing at a shitty shipping company for a sysadmin position responsible for two NT boxes, three QNX boxes and about 10 diskless stations where the workers scanned packages.

All connected via an ARCNET setup.

Using Zebra thermal printers.

I am certain that if hell exists it is a cold (below freezing), dusty world where your only job is to keep Zebra thermal printers working. I will never take a job again that involves those horrible devices.

I ended up getting the job and was just killing it (in a good way). By the end of my first year I had scripted most of our processes so that even the non-computer people could do some of the more mundane tasks like uploading inventory files, looking up packages in the database, etc which meant I could spend more time on the fun things. I ended up making our location the spot to send your jacked up Zebra printers to for repair and at 19 years old was being flown around to fill in for IT managers when they were on vacation. What was really amazing was that I had full health care, a 401k and was making just about twice as much a year as my mom had ever made. I don’t say that to be negative or to slam my mom, she did the best with what she had and managed to keep me fed and clothed, even if it did come from food stamps or government programs. The key takeaway is that because of technology, at 19 years old, I was already living an arguably better life than either of my parents had ever had.

Seriously, think about that for a minute and try to understand how much of a game changer that is, I’ll wait.

That single job was a foundation for a career that in my mid-thirties allows me to live comfortably and debt free, own a home, raise a child who has access to everything he needs and occasionally even do fun things like take vacations to another country. All of that with no formal education. What other industry would you ever find that in? I am not well versed in all of the other roles and trades, but I am willing to bet that it’s pretty rare. Even the more “blue collar” jobs typically require some type of apprenticeship (aka formal schooling) before you can actually start your career. Most jobs in the world of computers need either proof of formal education or experience. This experience is what has allowed me to climb the career ladder and get to a point that as a kid I never thought was possible. Do you know where that entry level education that allowed me to get that job came from? From those years as a teenager that I spent dicking around on the computer being a script kiddie.

Let’s take the serious level up a notch. Think about how much luck was involved in order for that computer to enter my home so many years ago. I am almost certain that had it not been for that computer I wouldn’t be living the lifestyle that I have now. Now think about how many other children do not have such luck and never get the chance to get in flame wars on IRC or read 2600. Are we shorting a huge section of our population and missing out on talent? I know people will say things like “you could have used computers at the library” or “you could have used school computers” but that is bullshit. The availability of both of those locations is pretty limited and typically locked down with regards to what you can access. People who are not born with the gift need to be able to immerse themselves in the world of computers with no roadblocks or barriers, that’s how we make up for the lack of natural talent. Also you can’t install python on those computers to make 1337 scripts.

Now comes the part where I talk about the purpose of this post and that is to ask: what are you doing to help evangelize the world of computers? Are you donating time and/or money to organizations that help people who might not typically have computer access get it? Are you giving your old (but still working) hardware to someone who could really use it? Are you explaining to your nieces and nephews not only what a payphone is but how you used to use a redbox to get free calls on those ancient artifacts?

Not everyone has to be a developer so think outside of that. Think about all of those DBAs out there or the desktop PC admins. What about the network engineers or the Active Directory people (god bless their souls)? Take the opportunity to help the kids who have no idea what the world of computers can offer them and help them realize what is possible. Go out and light a fire under the ones who are into it, show them cool shit! Setup a hack day at your work and invite schools from lower income areas to come in and hack on hardware or software. Show them some of the cool stuff you can do with computers and help them understand what is out there waiting for them if they want to work at it. If you have a story like mine share it! It’s great to talk about computers if you come from a privileged background (seriously, it is), but it’s even more impactful when you can talk about a story that relates to the world they live in. Shit, share it at your local users group or meetup in the hopes that someone else is inspired and starts helping. Have no clue where to start? My contact info is at the top of the page, contact me and I’d love to help.

Hand Wired Keyboard pt2

esell@esheavyindustries.com (esells) — Fri, 27 May 2016 21:10:30 -0600

Hand Wired Keyboard Part 2

At the end of part 1 I had a working keyboard but it had to be plugged in via USB. I haven’t used a wired keyboard in probably five years so having this one wired made me sad. There were a few hints at how people had done bluetooth + teensy setups (the teensy is the brains) but no real tutorials or anything. So as I typically do when presented with an electronics project, I headed over to adafruit to see how much I could get already pre-built for me.

I already had the battery/charging stuff from a previous project so all I needed was something for the bluetooth. Guess what? They have a part for that: bluefruit EZ-Key. This little guy has the ability to take input in on it’s UART port and fire it off via bluetooth, perfect!

Now I had to figure out the wiring as I wanted to be able to charge the battery via the teensy USB port as opposed to using the USB port on the lipo charger board. There is a small trace on the teensy that you can cut to stop the USB port from actually providing power (see here) which sounded like what I wanted so I had a few drinks to steady my hand, pulled out the xacto knife and made the cut. After that it was just a matter of getting the TX/RX from the teensy hooked up to the EZ-Key and then the power + charging coming from the lipo charger board to the teensy. Here is an amazing diagram I put together showing what I did:

pro schematic

The center 5v pad on the teensy is powered by the USB plug, so to charge the battery I just plug the keyboard up via USB to a power source and I’m good to go. What the awesome diagram doesn’t show is a small switch I added on the “power” wire between the charger board and teensy so that I can turn off the keyboard when I’m not using it.

This setup was functional and it worked correctly but because of the extra components (mainly the battery) the original spacers that came with the keyboard didn’t work. I tried about five different arrangements but every time I ended up pinching a diode or something causing the keyboard to do strange things on certain keys. The solution was to just order some taller spacers. This means that I will need to order a new middle piece cut from acrylic in order to hide all the guts but honestly I kind of like the exposed look.

With regards to the firmware, a few changes had to be made so I did my best to pretend to be a C programmer. The biggest change was renaming a bunch of macros that had at some point been renamed and never updated across the TMK code base, you can see the PR I submitted for that here. After that was done it was basically just a matter of copying the bluetooth converter sample project in TMK and updating it to fit my actual keyboard. You can see the changes I made here. Please keep in mind that the keymap being used is very basic and there is a lot of clean-up that could be done. This was simply just to verify everything was working as it should.:w

Hand Wired Keyboard pt1

esell@esheavyindustries.com (esells) — Fri, 22 Apr 2016 21:10:30 -0600

Hand Wired Keyboard Part 1

I had been toying with the idea of buying a mechanical keyboard for a few years but the bug didn’t really hit until about 8 months ago. Me being me I thought “why just buy one when I could build it myself?!“. It turns out that putting your own mechanical keyboard together is kind of a pain in the ass, that’s why.

Ok, so I’m going to build a keyboard, time to actually research what it takes to build a keyboard. As with most things, the keyboard community has it’s own terms, language and acronyms so I spent the majority of the first month trying to figure out what the hell they were talking about and mapping their lingo to the actual parts I would need. At this point I thought I had a pretty good idea on what I needed so I went out to source probably the most important part of a keyboard: the switches. These are what your key caps go onto and what actually send the computer a signal that something has happened.

made of baby parts

Turns out there are about 500 fucking kinds of switches out there and they all do different things just a tiny bit differently. Some are “linear”, some are “tactile bump” and some can even be “tactile click”. That is just a small sampling and those are the major categories, under those are tons of other categories that ends up creating this insane matrix of keyboard switches. I knew I wanted something that wasn’t super “clicky” because that is annoying so I was told to check out the “cherry mx brown” switch. This should have been the first sign of what I was getting into. Go out to Google/Amazon/eBay and try to find 60-80 cherry mx brown switches, I’ll wait. Turns out these things (as well as other cherry mx switches) are in such high demand that they are hardly ever available. When they do pop up they sell out super quick unless it happens to match-up with one of their big shipments that I guess only happen a couple times a year. Anyways, I was coming up empty handed everywhere I looked so I had to go the message board group-buy route. Luckily the group-buy was legit and I ended up with all the switches I needed.

Ah, so now I needed something to put those switches into aka the “plate”. I knew I wanted a ~65% style keyboard which is basically a typical keyboard stripped of all the non-essential stuff (for me). Things like function keys, media keys, etc. I came across a few sites (plate & case builder and keyboard layout editor) that allowed you to design your own keyboard layout and then create a CAD type file from those layouts which could then be provided to a shop or vendor who could cut the pattern out in metal, plastic, whatever. So I created my layout, created my CAD files for the plates (top and bottom) and headed over to big blue saw to get my quote. For one set of plates it was going to cost me ~$90!! Now I’m not saying that the pricing from big blue saw is out of line, it just wasn’t what I was expecting. Once you order five or more sets of the plates the pricing was almost cut in half but I wasn’t going to be able to find that many people who wanted to make their own keyboard.

By pure chance I came across someone online talking about their “neutrino” build. It was a 65% keyboard that was being sold by Ortholinear Keyboards and it was right in my price range so I went ahead and pulled the trigger. This ended up being yet another pain that I wasn’t anticipating as due to a previous group buy offered by Ortholinear Keyboards, they were so backed up with orders that my stuff didn’t ship for over four months. At this point I’m starting to realize that things in the mechanical keyboard game don’t come easy.

When the plates for the keyboard finally arrived I grabbed all the other parts I had ordered and went to town. Initially I had talked myself into believing that soldering all of these random diodes and wires wasn’t going to be an issue and that I’d knock it out in no time. wrong. wrong. Through hole soldering will never (never!) prepare you for the pain that is soldering a hand wired keyboard. If you are reading this right now because you are thinking about doing this I HIGHLY suggest you either make your own PCB or buy a kit that comes with one.

I stupidly sat down at my table, spread everything out in front of me and assumed in an hour I’d have a keyboard ready to go.

what you need for a keyboard

The suggested process is to put all of your switches in and then start wiring up the rows (which includes the diodes) and then the columns so that’s exactly what I did. It quickly became evident that there is no easy way to solder the diodes onto the switch and then to the other diode across from it. I tried about five different techniques and none of them really made it easy. Having a few drinks however did make me less angry so that is my suggestion if you plan on doing this: alcohol + hours of spare time. Between trying various ways to make it easier, throwing things and actual soldering it took me about 30 minutes to do the first row. If you’re keeping track you’ll realize that based on my initial time calculations that left me with 30 minutes to solder four more rows. I decided to just shut it down and work on the other rows later which in hindsight was probably the right call. The next day I was able to come back knowing what type of pain I was in for and just power through it. The result:

never again

Rows were done so that meant the columns were next. No diodes this time just straight up wire, no problem right? Nope. You don’t want the wiring to touch so that means you need to leave the shielding on the wiring you use for the columns. So to expose the wire I needed to solder I just used my pocket knife and stripped away small sections where I needed the bare wire exposed. That process took me way too long. The actual soldering was way easier than the rows though so I was able to cruise through that pretty quickly.

are we there yet?

“Oh god I’m so close at this point” is all I could think when it came time to wire up the teensy. The teensy is basically the brains of the keyboard and connected to the various rows and columns I had created earlier. Routing the wires was kind of a pain but nothing crazy and overall this was probably the least stressful part of the build.

so.damn.close.

At this point I attached the base plate but left out the middle acrylic piece because I needed access to the reset button on the teensy and had plans to re-work a few things anyways. I also put the actual key caps on and went to town creating the firmware needed to make this beast actually type. A group of people way smarter than I am have a great firmware project out on Github just for projects like this, it’s called TMK and is actually pretty easy to use. Since all keyboards are going to be different, TMK has you pass in what your keyboard matrix looks like and then what you want each key to do when it’s pressed. TMK also has this concept of layers which is great for my 65% board as I can still have the functionality of say, media keys, I just need to use a specific key sequence to activate it. As an example I don’t have a “tilde” key by default so if I press fn + Esc I’ll get a tilde. Anyways, I got the firmware to compile and loaded it onto the teensy and it actually worked! Below is the final keyboard with a few keys that I’ll have to swap out as they are not the right size/type:

tada!

This is where I’m at currently but the project is not complete. The idea of having to use a wired keyboard in 2016 makes me sad so the next phase will be to hook in a bluetooth + battery setup so I can use this POS without an ugly USB cable attached to it. For now though I’m going to take some time to recover.

nixie clock part 1-a

esell@esheavyindustries.com (esells) — Thu, 24 Mar 2016 21:10:30 -0600

Nixie clock part 1-a

So things have changed. I’ve taken out the Arudino and replaced it with an Adafruit Pro Trinket. Additionally I updated the PCB so show the Adafruit Bluefruit and took out the homebrew RTC circuit and just went with the Adafruit DS3231 breakout.

still looks like shit

The reason for these changes was mainly just for simplicity. The new parts are easier to source, come from a vendor who I know makes really good/cool stuff and it makes my life a bit easier. Now I’m off to eBay to start the search for nixie tubes!

nixie clock part 1

esell@esheavyindustries.com (esells) — Sat, 19 Mar 2016 19:16:20 -0600

Nixie clock part 1

I enjoy building things and for some reason, I tend to enjoy building more “classic” things. Awhile back I came across the Adafruit Ice Tube Clock Kit which I thought was one amazing looking clock. Unfortunately they discontinued it but since I enjoy making things harder on myself than they should be, I went to Google to try and find something similar. What I came across was a pretty popular project called a nixie clock.

example of a nixie clock

A nixie clock is simply a clock that is built using multiple nixie tubes for the display. They come in all sorts of different sizes and shapes which makes it easy to design a clock for whatever purpose you want (small desk clock, larger wall clock, etc). In my case I wanted something similar to the Adafruit clock, small enough that I could put it on a desk but still big enough to show off some of the components. I like the mechanical look so I had planned on putting everything in some type of clear case so you could see all of the electronics.

Research

I am not an electronics guy at all, I can take a bag of parts with a schematic or instructions and solder it all together, but if you are looking at me to design a circuit from the ground up then you are in for a world of hurt. Luckily there are a bunch of smart people out there that have built nixie clocks before and have been kind enough to share. One of those people is Andrew Saxén who has not only a great write-up on how he built his nixie clock, but also links to the code that runs it as well as the PCB layout. You can check it out at his site Code Terrific. There are tons of other links out there showing various builds, but for whatever reason Adam’s just seemed to click with me so I decided I was going to use what he had already done as my “base”.

chop chop

For my project all I wanted was a clock, I didn’t really care about temp or humidity so there wouldn’t be a need to include a DHT22 sensor in the build. Additionally, as of right now I plan on having the PCB printed as opposed to etching one at home. This means I’ll be able to take advantage of using a two sided PCB. The third change from Adam’s original design I’d like to make is to directly integrate his real-time clock circuit into the nixie PCB, that way everything is all in one spot.

With all of that I downloaded the Fritzing application and set out to re-create Adam’s PCB as best I could. Once I had that I started to strip away things I didn’t think I’d need, add some new things and then try and condense it down as much as possible to save on costs. Part of me wanted to make a really nice looking PCB but for my first attempt the cost would have been a bit more than I wanted to spend. I’m thinking if this turns out ok I can always re-use the nixie tubes and create a new, fancier PCB later.

holy rats nest!

If your skills are at hobbyist electronics person or above I apologize for the mess that is the PCB above :)

With this design it would be about ~$58 for three boards from OSH Park which isn’t that bad for me. With a small child around and a full-time job the ability for me to find time to hand etch these is highly unlikely so it makes more sense to just bite the bullet and have someone else print them.

Next steps

The base PCB isn’t finalized but I think it’s close enough for now. The next thing that I have been thinking about is how to mount the tubes. Part of me would like to make an additional PCB and have the tubes slot right into that PCB and then connect it to the other PCB either via headers or a ribbon cable. The problem with that is since I’m pretty much useless, I currently don’t have the ability to design the libraries needed to mock out IN-8 tubes and I haven’t been able to find any existing resources. Everyone seems to have the IN-12A but I just don’t like the look of those as much.

Another option would be to use these pre-built “sockets” from pv electronics. Outside of having a nice way to interface with the tubes though I’m not sure how much they would get me.

The third option is do just do what Adam did and run the wires directly to the tubes. I think if I could do this in a fairly clean manner it would look pretty cool and go with the whole mechanical/exposed look I want.

Have some extra cash? Help make your life easier

esell@esheavyindustries.com (esells) — Sat, 13 Feb 2016 13:02:40 -0700

If you have some extra cash maybe you would consider donating some of it to one of the various open source projects that you likely use almost every single day? I know it is easy to forgot about where many of the things we use on a daily basis come from, but odds are that if you are reading this you probably use some kind of open source app. Where would you be without OpenSSH to securely connect to your servers? I know I’d personally be fucked without HAProxy doing the load balancing for many of the apps I’ve supported. What about Apache? Sure it might not be the hot tech now but how many of you have used it as your HTTP server both at work or for your personal site?

I could go on and on but the point I’m trying to make is that there are many (many!) tools that we use on a daily basis that someone (or a group of someones) are building, fixing and maintaining that make our lives easier. Some of these have big corporate sponsors but many of them do not. A lot of these people have other jobs they use to pay the bills and then work on their open source projects during their “off” time. Some groups build products, others focus on your rights (take the EFF for example). Others, like the Tor project, do both.

I encourage you to make a list of some of the open source tools you use on a daily basis and consider donating either money or time to those projects.

deb-simple, a no frills apt repository server

esell@esheavyindustries.com (esells) — Wed, 03 Feb 2016 01:46:49 -0700

I recently needed an simple way to serve up debian packages via apt-get for our internal apps. Oddly enough finding something that fit our needs was a bit harder than I thought. We needed something that:

Supported multiple versions of the same package
Didn’t care about signing or “changes” files
Allowed us to upload via an API or some type of incoming directory
Could handle multiple packages coming in at once without corrupting the repo
Was easy to setup/maintain

There were a few services out there that fit our needs but they were not free so I took a shot at building an app to fit our use-case. The outcome was deb-simple.

Currently it supports uploading packages via POST, deleting packages via DELETE and does a good job (I think) of making sure the repository listing doesn’t become corrupted.

Contact

esell@esheavyindustries.com (esells) — Wed, 18 Mar 2015 23:06:53 -0600