Categories

A collection of 5 things from the week that I found interesting, am enjoying, or am working on.

This week:

• I’ve been learning about: Single Sign-On and Sign-Out in .NET Core with OpenID Connect and IdentityServer4
• I’ve been watching: Andrew Tokely’s talk on High Performance Product Leadership
• I’ve been happy to see that: Webstock is back on this year
• I’ve been thinking about: Lambda School sounds great
• I’ve been playing: The Witness

Using Git on Windows is way easier once you find a combination of tools that you’re comfortable with.

In this blog post you’ll learn about:

• Using Git via the command line on Windows
• A great Git GUI client for Windows and Mac
• The tool that I use for diffs, and resolving any Git merge conflict that may occur (including 3-way merges)
• Using Git with SSH authentication on Windows via .bashrc and Bash shell
• My .gitconfig

Impromptu speaking in front of an audience is challenging for most of us. Even the famous author and public speaker, Mark Twain, has been quoted as saying “I never could make a good impromptu speech without several hours to prepare it.”

If you struggle with scattered thoughts, or mind blank when impromptu speaking, then this blog post will help you.

In this blog post, you’ll learn some techniques to structure your speech when you’re on the spot, and some ways to quickly generate ideas when your mind goes blank.

In this blog post, you’ll learn how to host a sample Python web app on Elastic Beanstalk, using HTTPS, without a load balancer.

After following the steps in this blog post, you should end up with something that looks like the following image:

A collection of 5 things from the week that I found interesting, am enjoying, or am working on.

This week:

• The Competitive Advantage of Design (Philip Fierlinger)
• How to get rich (without getting lucky)
• Startups that are hiring remotely in 2019
• 15 Things You Should Know About Product Managers
• How website builders can differentiate themselves in a heavily commoditised market

A collection of 5 things from the week that I found interesting, am enjoying, or am working on.

This week:

• Sometimes Netflix preview images and descriptions get out of sync, creating amusing mashups
• Troy Hunt’s tips on personal finance for technology professionals
• Patrick McKenzie on: Should you choose a commercial license or donation model to monetise your open source project?
• A great thread discussing when to practice TDD (Test Driven Development)
• Blender 3D tutorial: Create a Modern Bedroom in Blender in 35 Minutes

In this blog post I’ll show you how to scrape Income Statement, Balance Sheet, and Cash Flow data for companies from Yahoo Finance using Python, LXML, and Pandas.

I’ll use data from Mainfreight NZ (MFT.NZ) as an example.

The screenshot below shows what you can expect to get by following the steps in this blog post:

By following the steps in this blog post, you’ll also generate a DataFrame containing data from the Income Statement, and Cash Flow statement.

After creating Pandas DataFrames, I’ll show you how to export everything to an Excel file, so you’ll have output that looks something like this:

One of the books that I read over the summer break was “Are we there yet? Insights on How to Lead by Design” by Sam Bucolo.

For the first few years of Sam’s career, he worked as a design consultant. As he gained more design and problem solving experience, he came to realise that he was often getting the wrong brief from his clients.

Sam found that while companies could conceptualise new products or services, the sticking point was aligning this conceptualisation to their business models, which meant that time, and time again, projects significantly fell below expectation or failed completely.

These experiences led Sam to specialise in Design Led Innovation.

My notes:

There are three elements of Design Led Innovation:

1. Customer value: This is the starting point, and is about being clear on who the customer is, and the problem that the business is solving for them.
2. Management Mindset: The correct mindset to create a strategy which has been built around new insights about the customer.
3. Strategic alignment: Ensuring that all business processes and systems are aligned with the customer’s problem.

I’ve been using Hugo for my blog for the last few years, and recently published my first theme - Silhouette Hugo.

I recently received an email with the following question:

What is the simplest way for me to rename the categories Development and Golang into other names, like Projects and Gallery? Is this in partials or template or theme or where?

I figured that this might be a common question, so decided to share it in a blog post.

Here’s how to rename categories in Hugo, in the context of my theme.

A collection of 5 things from the week that I found interesting, am enjoying, or am working on.

In this blog post, you will learn how to create a Python app using Flask and the Google API which will:

• Support Google Authentication with Python and Flask
• Restrict access via an OAuth scope, so that the app can only view and manage Google Drive files and folders which were created by the app
• Read and write files on the user’s Google Drive with Python.

By the time you get to the end of this blog post, you’ll have built a basic Google Drive file browser which looks something like:

This blog post is divided up into sections which progressively build up an app which interacts with the user’s Google Drive. If you only want to find out about how to do user Authentication with Google and Python, feel free to stop there.

A collection of 5 things from the week that I found interesting, am enjoying, or am working on.

I recently attended a presentation by BNZ’s Chief Economist, Tony Alexander.

One of the topics that really caught my attention was his perspective on the things that influenced the housing market over the decades, from the 1950’s until today.

I wrote up a bunch of notes to summarise his points.

If you’re interested in getting a bit of perspective on the NZ property market, and some of the things that caused it’s near-relentless movement upward over time, then read on.

If you attempt to read encrypted data from a DynamoDB table which has been restored from a backup to a DynamoDB table which doesn’t match the original table name, you may see the following errors:

TL;DR the restored DynamoDB table must have the same name as the original DynamoDB table, and be restored to the same account that it was originally created in.

The recommended way to enable HTTPS in Elastic Beanstalk is to use one of AWS’s load balancers such as the Application Load Balancer (ALB) which supports autoscaling, fault tolerance, and other things.

This blog is about hosting a web app prototype on a single EC2 instance, using HTTPS via Let’s Encrypt, without a load balancer.

Using an AWS ALB costs a minimum of about $18 per month, on top of any other charges you currently have, such as $5 for the t2.micro instance that you may be running your prototype on.

So, if you’ve only got one EC2 instance in Elastic Beanstalk for your prototype, and don’t currently want the benefits of an ALB (fault tolerance, auto-scaling, etc), but do want the benefits of HTTPS (protection from interception, man-in-the-middle (MITM) attacks, etc), read on.

When I first started using CKEditor 5 with create-react-app, I installed CKEditor as an npm module, and imported the ClassicEditor build as recommended by the quickstart.

Development mode (via npm start) worked well, and I was happily integrating CKEditor with React, but as soon as I ran npm run build (which generates the create-react-app production build), I ended up with the following error:

> [email protected] build c:\Dev\scratch\ckeditor-integration
> node scripts/build.js

Creating an optimized production build...
Failed to compile.

Failed to minify the code from this file:

        ./node_modules/@ckeditor/ckeditor5-build-classic/build/ckeditor.js:5:7350

Read more here: http://bit.ly/2tRViJ9

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `node scripts/build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Ruh-roh. What to do next? How can I solve this? Is this solvable? Is there another way?

I completed the AWS Solutions Architect Associate exam in November 2017, after about eight weeks study, with a score of 85%.

Below are a series links to blog posts containing all my study notes, and finally, another blog post containing my thoughts on the exam, as well as some exam feedback and tips for those who will sit the exam after me.

I’ve been hosting this blog on Amazon S3 for a few months now, and thought it would be a good idea to write about how it’s set up, in case future me needs to refer to it, and to give pointers to anybody else who is interested in setting up their own website using Amazon S3, HTTPS, and the CloudFlare CDN.

Hosting this website costs cents per month, and avoids security risks of other blog platforms such as Wordpress by being purely static, consisting just regular HTML files, JavaScript, and images; which means no PHP, .NET, etc. This works well with what I need in a blog.

After heaps of study, I passed AWS Solutions Architect Associate exam, with an overall score of 85%.

Here are my notes on how I studied for the exam, and tips for preparing based on my experience in the actual exam.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

These notes were written while working through the A Cloud Guru AWS Certified Solutions Architect - Associate online course. These notes are partly from the videos, and also from various other online sources. Primarily, they’re notes for me, but you might find them useful too.

Since the AWS platform is changing so quickly, it’s possible that some of these notes may be out of date, so please take that into consideration if you are reading them.

Fiddler is an awesome tool for debugging web services. For the last 10 years, I’ve been using it virtually every working day for web service debugging in scenarios such as:

• Capturing HTTP requests as they travel through a distributed system, which you have set up on your local development machine.
• Intercepting and modifying HTTP requests, essentially performing a local man-in-the-middle interception. This is particularly useful if want to modify HTTP requests to simulate functionality which isn’t currently present, modify HTTP headers, etc.
• Replaying captured requests - useful if you are debugging the way a web service behaves, given a specific request; capture the request, and replay it as many times as you need in order to debug the error.

If you’re doing any work with web services, Fiddler is definitly worth considering. Setting it up is fairly straightfoward

By default, Fiddler can update your browser proxy to route through the Fiddler proxy, which is, by default on port 8888, and will capture requests between Chrome/Firefox/Safari/etc, and your web app.

What if you’re debugging locally, and want to intercept .NET services which call other web services? i.e. Web Browser -> Web App -> Service1 -> Service2 (the one you want to debug)? This can be done by setting defaultProxy in web.config. In this case, since we want to debug Service2, we’d set defaultProxy in Service2’s web.config file.

i.e.

<configuration>
 <system.net>
  <defaultProxy>
   <proxy bypassonlocal="false" usesystemdefault="true" />
  </defaultProxy>
 </system.net>
</configuration>

With one of the distributed apps I frequently work with, I was setting, and resetting the web.config defaultProxy fairly frequently. To save some effort, I wrote a script to automate the change.

A collection of links, with a Wellington focus, which I frequently use to work out if it’s worth going for a dive or not.

Feel free to get in touch if you have any links worth adding.

Webcams

Beacon Hill Webcam

Cornish - Lyall Bay webcam

Victoria University Coastal Ecology Lab - South coast time-lapse camera

Forecasts

MetService - 5 day forecast for Wellington

MetService - 10 day forecast for Wellington

Swellmap - Surf Forecast Conditions and Marine Weather

White Cloud Kiteboarding - Wind report

Windy - Interactive wind forecast maps

Maps

Cawthron Eye - Satellite view

LINZ - Hydrographic raster chart

Water Quality

Shellfish Biotoxin Alerts

Wellington Water Quality

Earlier this evening I installed .NET core preview 2 on Debian Sid, and tried to create a new project via the dotnet new command, only to get a segmention fault error message:

[email protected]:~/git/testproject$ dotnet new console
Segmentation fault

In this instance, the segmentation fault on creation of a new project was is due to .NET Core telemetry being incompatible with version 1.1 of OpenSSL.

Sometimes, when being interviewed for a job as a software developer, you’ll be asked a question such as “write a method that takes an integer as a parameter, and returns it’s factorial.”

For example, the factorial of 3 is represented as “3!”, which is calculated via 3*2*1, which equals 6. 4! is 4*3*2*1, which is 24, etc.

Putting aside whether these kinds of questions should be asked in an interview, if they’re asking you this, there’s a fairly high likelihood they’re asking for you to show that you understand recursion. If that’s the case, no problem, something like this will calculate the factorial:

let rec factorial n =
    match n with
    | 0 | 1 -> 1
    | n when n > 0 && n <= 12 -> n * factorial (n - 1)
    | _ -> failwith "Parameter n is out of the supported range. Must be between 0 and 12."

This can be run in the F# interactive shell via:

> factorial 3;;
val it : int = 6

But what if they want to find out whether you:

1. Understand recursion, and…
2. Know when you can avoid recursion, and just write simple methods instead.

I recently migrated my mattbutton.com blog away from Wordpress hosting in favor of a static site generated by Hugo, hosted on Amazon S3.

Initially, I hosted the static site via Aerobatic.io, who recently removed their free tier, and started charging $15 per month for hosting with a custom domain.. 50% more than the $10 per month I was previously paying for Hostgator Wordpress hosting. Unless I have certain specific requirements, I can’t justfy that kind of cost to host a static site.

Aerobatic.io are using Amazon S3 and Amazon CloudFront behind the scenes, so I decided to cut out the middle-man, set it up myself, and save nearly $15 per month.

Setting up the S3 bucket to host my site was fine. For the CDN/SSL side of things I initially tried using CloudFront because most of the AWS Hugo Hosting, HowTo guides were using it.

When trying to set up CloudFront via my personal AWS account, I got an error saying a distribution already exists for mattbutton.com. The reason for this error is because Aerobatic.io had already created a CloudFront distribution pointing to their own S3 bucket. CloudFront isn’t an option for me until Aerobatic.io delete their mattbutton.com CloudFront distribution.

I still wanted to use SSL, and had decided on setting up mattbutton.com with it mainly out of interest, partly because Google uses HTTPS as a ranking signal, and partly because Chrome will eventually show a Not Secure warning for all pages served over HTTP.

Since AWS CloudFront wasn’t an option, I decided on using the CloudFlare free plan for SSL and CDN. Everything went well, until I encountered a CloudFlare 502 Bad Gateway error page:

I wasn’t having any luck searching for a solution to this error for this particular error. Fortunately, there’s a simple fix, if you know what you’re looking for, and you’re happy with the trade-offs involved.

I joined a Toastmasters public speaking club back in November 2016. There are lots of resources out there, I found they were fragemented, and none really resonated my personal style of speaking, especially for the club I’m in, which adheres to the Toastmasters structure and rules, yet has a much more natural speaking than other clubs I’ve visited.

After performing each of the roles several times (Chair, Grammarian, Table Topics Master, etc) I found it useful to build up, and revise templates as my skill and personal style evolved. Below are my current templates, which I will update over time:

• Toastmasters Grammarian Template

I spent two weeks in late August on the island of Koh Tao, which is located in the eastern Gulf of Thailand.

The weather was perfect. Every day, I’d head out on the boat to go scuba diving at various dive sites around the island. I’d been a qualified diver for several months prior, so no training was needed, and I signed up for the fun dives, where we would be taken on underwater tours by a guide called a dive master.

I was planning on diving virtually every day around the beautiful island of Koh Tao. In particular, I was hoping to see some big sea creatures such as the Whale Shark.

Whenever I travel, I keep my luggage as light as possible, traveling with carry-on only; everything fits inside a 38 litre backpack, usually with room to spare, and this trip was no exception.

My entire set of luggage, for my 2016, six week trip around Japan and Thailand, weighed about 6kg (14lb). During my trip, I picked up a few more things, which pushed the total weight up to about 7kg, which was still within the carry-on weight allowance.

When you want to visualize how a stock or portfolio has performed historically relative to the market as a whole, it is useful to create a comparison chart.

This blog shows how to create a line chart to compare two stocks with Deedle, FSharp Charting and F# Data.

In this example, the chart will show the perfomance of ANZ.AX relative to the ASX ALL ORDINARIES index ([^AORD]5) over a three year period from 2011-1-1 to 2014-1-1.

This article is written for the intermediate F# audience who has a basic familiarity of stock trading and technical analysis, and is intended to show the basics of implementing a backtesting system in F#.

If you’re an F# beginner it may not take too long for you to get up to speed on the concepts if you check out a few resources. In particular:

• Try F#
• The CSV Type Provider
• Mastering F# Lists
• and Wikibooks F Sharp Programming/Recursion

The backtesting strategy which you will implement is a simple SMA crossover where the fast line is 10 bars and the slow one 25. When the fast line crosses above the slow line, a buy signal will be triggered, and when the fast line cross below the slow line, a sell signal will be triggered. Only one long position will exist at any time, so the system will not trigger another buy order until after the long position is sold.

When migrating a site from ASP.NET MVC 1 to ASP.NET MVC 2, you can generally follow the instructions in http://www.asp.net/whitepapers/what-is-new-in-aspnet-mvc#_TOC2, taking note of any breaking changes. This will take you most of the way there, however there are a few undocumented issues which you may uncover if you’re migrating a site with a substantial amount of code.

By default, the ASP.NET MVC 1 model binder would initialize strings to string.Empty whereas ASP.NET MVC 2 will initialize strings as NULL. This is an undocumented breaking change and will be a problem if you have a substantial amount of code relying on the original behavior – code that was previously working in production will start throwing NullReferenceException.

To preserve the original MVC 1 model binder behaviour, consider creating default model binder such as the following:

For the ASP.NET MVC default model binder to bind a list successfully, the list must be sequential with unbroken indices.

For the following examples, the server side model will be

public class ItemsModel 
{
    public IList<string> Items { get; set; }
}

Non-sequential form submits when using indexes which don’t start from zero i.e. Item[2], Item[3], etc will result in incomplete form data being loaded by the model binder.

The following will not bind, and will result in the model being NULL:

<input type="text" name="Item[1].Name" value="Item1" />
<input type="text" name="Item[3].Name" value="Item2" />
<input type="text" name="Item[4].Name" value="Item3" />

When you are maintaining multiple web apps across environments it can be difficult to keep track of which scripts need to be run to upgrade the database when it comes time to deploy. If you’re maintaining different versions of web apps across environments when the versions can sometimes be significantly out of sync, the difficulty of determining which update scripts need to be run on deployment can explode.

While there are a ton of approaches to keeping your database under version control, if you want something simple and effective that you can implement with minimal time and effort, consider a DatabaseVersionHistory table in your database.

A database version history table will allow you to see at a glance the state the database is in and by comparing it with the update scripts in source control, you will quickly be able to determine which update scripts need to be run.

After installing Visual Studio 2012, I found that the open file dialog wasn’t being displayed. I could open projects via Windows explorer, via the recent projects menu, compile, run etc, however neither Open Project or the File->Open->Project/Solution were working.

What was strange about this is that other dialogs such as New Project were working fine.

After much searching and testing, enabling the Tablet PC Input Service fixed the issue. This does not make much sense since I’m not using a tablet pc, however it works for me and may work for you.

To enable the service:

Why would I want to use Mercurial or any other DVCS client with a Subversion repository?

• It lets us keep SVN as our central repository
• Some team members prefer not to use a DVCS for whatever reason so it lets them carry on using SVN without interruption.
• It allows me to work and commit changes (but not push!), search history and switch between branches completely disconnected. I can continue to work during network outages or while traveling when I don’t have connectivity.
• You get full, fast history search.
• Switching between branches is easy and fast.
• Any automated processes which use SVN (i.e. automated builds and deployments) can continue to operate while everyone moves to DVCS.
• It’s much easier to perform merges than regular SVN (via export/import patch queues – which I detail later)

If you are connecting to a SQL server with the SQL profiler and none of your templates are showing up, compare the versions of the SQL profiler you are running and the version of SQL server that you’re connecting to; there is likely a version mismatch.

If this is the case, what’s likely happening here is that you’re connecting to a SQL 10.50 instance with a SQL 10.0 profiler and the profile templates for 10.50 aren’t present.

When running a page with HTTPS, you’ll want to also load any external resources such as javascript via HTTPS. A lot of people recommend loading jQuery from the Google CDN via the following javascript script:

<script type="text/javascript">
    var gaJsHost = (("https:" == document.location.protocol) ? "https://" : "http://");
    document.write(unescape("%3Cscript src='" + gaJsHost + "ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js' type='text/javascript'%3E%3C/script%3E"));
    document.write(unescape("%3Cscript src='" + gaJsHost + "ajax.googleapis.com/ajax/libs/jqueryui/1.7.2/jquery-ui.min.js' type='text/javascript'%3E%3C/script%3E"));
    document.write(unescape("%3Cscript src='" + gaJsHost + "ajax.googleapis.com/ajax/libs/swfobject/2.1/swfobject.js' type='text/javascript'%3E%3C/script%3E"));       
</script>

This works just fine, however you can let the browser select the protocol depending on the request by the following snippet:

We used to use Balsamiq for mockups, however Expression Blend 4 and Sketchflow has some compelling features.

Sketchflow is missing a sketch style grid among other things, so we looked to Sketchables to provide.