Software Development

A 39-post collection

13 years ago we were rallying against nested HTML tables - Notes from Webstock 2019

By Matt Button |  Jun 29, 2019  | webstock, notes

I’m not sure why it’s taken so long, but after 13 years of Webstock existing, this year was the first time that I attended.

While I haven’t attended Webstock until now, I have watched many of the recordings which are usually available a few months after each event was held, and it’s been interesting to see how the event, and the industry in general has changed over the years.

None of this year’s talks were technical, and all were about the human side of the industry - topics such as ethics, personal experiences, psychology, communication, and team health.

One of the original organisers stood up on stage at the end of the event, and said it best:

13 years ago we were rallying against nested HTML tables

Continue Reading...

Using Git on Windows

By Matt Button |  Mar 24, 2019  | git

Using Git on Windows is way easier once you find a combination of tools that you’re comfortable with.

In this blog post you’ll learn about:

  • Using Git via the command line on Windows
  • A great Git GUI client for Windows and Mac
  • The tool that I use for diffs, and resolving any Git merge conflict that may occur (including 3-way merges)
  • Using Git with SSH authentication on Windows via .bashrc and Bash shell
  • My .gitconfig
Continue Reading...

How to scrape Yahoo Finance and extract fundamental stock market data using Python, LXML, and Pandas

By Matt Button |  Jan 24, 2019  | python, pandas, lxml, scraping, finance, featured

In this blog post I’ll show you how to scrape Income Statement, Balance Sheet, and Cash Flow data for companies from Yahoo Finance using Python, LXML, and Pandas.

I’ll use data from Mainfreight NZ (MFT.NZ) as an example.

The screenshot below shows what you can expect to get by following the steps in this blog post:

The first few columns of a Pandas DataFrame containing MFT.NZ Balance Sheet data

By following the steps in this blog post, you’ll also generate a DataFrame containing data from the Income Statement, and Cash Flow statement.

After creating Pandas DataFrames, I’ll show you how to export everything to an Excel file, so you’ll have output that looks something like this:

Continue Reading...

How to rename categories in Hugo

By Matt Button |  Jan 13, 2019  | hugo

I’ve been using Hugo for my blog for the last few years, and recently published my first theme - Silhouette Hugo.

I recently received an email with the following question:

What is the simplest way for me to rename the categories Development and Golang into other names, like Projects and Gallery? Is this in partials or template or theme or where?

I figured that this might be a common question, so decided to share it in a blog post.

Here’s how to rename categories in Hugo, in the context of my theme.

Continue Reading...

Google Authentication with Python and Flask

By Matt Button |  Jan 5, 2019  | python, flask, google-authentication, google-api, google-drive, featured

In this blog post, you will learn how to create a Python app using Flask and the Google API which will:

  • Support Google Authentication with Python and Flask
  • Restrict access via an OAuth scope, so that the app can only view and manage Google Drive files and folders which were created by the app
  • Read and write files on the user’s Google Drive with Python.

By the time you get to the end of this blog post, you’ll have built a basic Google Drive file browser which looks something like:

OAuth Google Drive Scope

This blog post is divided up into sections which progressively build up an app which interacts with the user’s Google Drive. If you only want to find out about how to do user Authentication with Google and Python, feel free to stop there.

Continue Reading...

InvalidCypherTextException when reading an encrypted DynamoDB table which has been restored from a backup

By Matt Button |  Dec 5, 2018  | aws, dynamodb, encryption

If you attempt to read encrypted data from a DynamoDB table which has been restored from a backup to a DynamoDB table which doesn’t match the original table name, you may see the following errors:

TL;DR the restored DynamoDB table must have the same name as the original DynamoDB table, and be restored to the same account that it was originally created in.

Continue Reading...

Enabling HTTPS on Elastic Beanstalk without a load balancer

By Matt Button |  Aug 6, 2018  | aws, alb, elastic-beanstalk, https, prototype

The recommended way to enable HTTPS in Elastic Beanstalk is to use one of AWS’s load balancers such as the Application Load Balancer (ALB) which supports autoscaling, fault tolerance, and other things.

This blog is about hosting a web app prototype on a single EC2 instance, using HTTPS via Let’s Encrypt, without a load balancer.

Using an AWS ALB costs a minimum of about $18 per month, on top of any other charges you currently have, such as $5 for the t2.micro instance that you may be running your prototype on.

So, if you’ve only got one EC2 instance in Elastic Beanstalk for your prototype, and don’t currently want the benefits of an ALB (fault tolerance, auto-scaling, etc), but do want the benefits of HTTPS (protection from interception, man-in-the-middle (MITM) attacks, etc), read on.

Continue Reading...