Software Development

A 35-post collection

How to scrape Yahoo Finance and extract fundamental stock market data using Python, LXML, and Pandas

By Matt Button |  Jan 24, 2019  | python, pandas, lxml, scraping, finance, featured

In this blog post I’ll show you how to scrape Income Statement, Balance Sheet, and Cash Flow data for companies from Yahoo Finance using Python, LXML, and Pandas.

I’ll use data from Mainfreight NZ (MFT.NZ) as an example.

The screenshot below shows what you can expect to get by following the steps in this blog post:

The first few columns of a Pandas DataFrame containing MFT.NZ Balance Sheet data

By following the steps in this blog post, you’ll also generate a DataFrame containing data from the Income Statement, and Cash Flow statement.

After creating Pandas DataFrames, I’ll show you how to export everything to an Excel file, so you’ll have output that looks something like this:

Continue Reading...

How to rename categories in Hugo

By Matt Button |  Jan 13, 2019  | hugo

I’ve been using Hugo for my blog for the last few years, and recently published my first theme - Silhouette Hugo.

I recently received an email with the following question:

What is the simplest way for me to rename the categories Development and Golang into other names, like Projects and Gallery? Is this in partials or template or theme or where?

I figured that this might be a common question, so decided to share it in a blog post.

Here’s how to rename categories in Hugo, in the context of my theme.

Continue Reading...

Google Authentication with Python and Flask

By Matt Button |  Jan 5, 2019  | python, flask

This blog post is part 1 of a multi-part series, and covers the basics of getting Google authentication to work with Python and Flask. Part 2 will cover actually reading and writing the files to Google Drive.

In this blog post, you will learn how to create a Python Flask app, which allows a user to log in with their Google account

Continue Reading...

InvalidCypherTextException when reading an encrypted DynamoDB table which has been restored from a backup

By Matt Button |  Dec 5, 2018  | aws, dynamodb, encryption

If you attempt to read encrypted data from a DynamoDB table which has been restored from a backup to a DynamoDB table which doesn’t match the original table name, you may see the following errors:

InvalidCiphertextException: An error occurred (InvalidCiphertextException) when calling the Decrypt operation

and

UnwrappingError: Failed to unwrap AWS KMS protected materials

TL;DR the restored DynamoDB table must have the same name as the original DynamoDB table, and be restored to the same account that it was originally created in.

Continue Reading...

Enabling HTTPS on Elastic Beanstalk without a load balancer

By Matt Button |  Aug 6, 2018  | aws, alb, elastic-beanstalk, https, prototype

The recommended way to enable HTTPS in Elastic Beanstalk is to use one of AWS’s load balancers such as the Application Load Balancer (ALB) which supports autoscaling, fault tolerance, and other things.

This blog is about hosting a web app prototype on a single EC2 instance, using HTTPS via Let’s Encrypt, without a load balancer.

Using an AWS ALB costs a minimum of about $18 per month, on top of any other charges you currently have, such as $5 for the t2.micro instance that you may be running your prototype on.

So, if you’ve only got one EC2 instance in Elastic Beanstalk for your prototype, and don’t currently want the benefits of an ALB (fault tolerance, auto-scaling, etc), but do want the benefits of HTTPS (protection from interception, man-in-the-middle (MITM) attacks, etc), read on.

Continue Reading...

Using CKEditor 5 with React via create-react-app

By Matt Button |  May 20, 2018  | react, ckeditor, npm, webpack

When I first started using CKEditor 5 with create-react-app, I installed CKEditor as an npm module, and imported the ClassicEditor build as recommended by the quickstart.

Development mode (via npm start) worked well, and I was happily integrating CKEditor with React, but as soon as I ran npm run build (which generates the create-react-app production build), I ended up with the following error:

> [email protected] build c:\Dev\scratch\ckeditor-integration
> node scripts/build.js

Creating an optimized production build...
Failed to compile.

Failed to minify the code from this file:

        ./node_modules/@ckeditor/ckeditor5-build-classic/build/ckeditor.js:5:7350

Read more here: http://bit.ly/2tRViJ9

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `node scripts/build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Ruh-roh. What to do next? How can I solve this? Is this solvable? Is there another way?

Continue Reading...

How I host and update www.mattbutton.com on AWS with CloudFlare and Amazon S3 Static Website Hosting

By Matt Button |  Nov 4, 2017  | aws, cloudflare, s3

I’ve been hosting this blog on Amazon S3 for a few months now, and thought it would be a good idea to write about how it’s set up, in case future me needs to refer to it, and to give pointers to anybody else who is interested in setting up their own website using Amazon S3, HTTPS, and the CloudFlare CDN.

Hosting this website costs cents per month, and avoids security risks of other blog platforms such as Wordpress by being purely static, consisting just regular HTML files, JavaScript, and images; which means no PHP, .NET, etc. This works well with what I need in a blog.

Continue Reading...