Featured

A 10-post collection

How to scrape Yahoo Finance and extract fundamental stock market data using Python, LXML, and Pandas

By Matt Button |  Jan 24, 2019  | python, pandas, lxml, scraping, finance, featured

In this blog post I’ll show you how to scrape Income Statement, Balance Sheet, and Cash Flow data for companies from Yahoo Finance using Python, LXML, and Pandas.

I’ll use data from Mainfreight NZ (MFT.NZ) as an example.

The screenshot below shows what you can expect to get by following the steps in this blog post:

The first few columns of a Pandas DataFrame containing MFT.NZ Balance Sheet data

By following the steps in this blog post, you’ll also generate a DataFrame containing data from the Income Statement, and Cash Flow statement.

After creating Pandas DataFrames, I’ll show you how to export everything to an Excel file, so you’ll have output that looks something like this:

Continue Reading...

A Basic Stock Trading Backtesting System for F# using Ta-Lib and FSharp.Data

By Matt |  Mar 8, 2014  | backtesting, f, f#-data, fsharp.data, ninjatrader, ta-lib, featured

This article is written for the intermediate F# audience who has a basic familiarity of stock trading and technical analysis, and is intended to show the basics of implementing a backtesting system in F#.

If you’re an F# beginner it may not take too long for you to get up to speed on the concepts if you check out a few resources. In particular:

The backtesting strategy which you will implement is a simple SMA crossover where the fast line is 10 bars and the slow one 25. When the fast line crosses above the slow line, a buy signal will be triggered, and when the fast line cross below the slow line, a sell signal will be triggered. Only one long position will exist at any time, so the system will not trigger another buy order until after the long position is sold.

Continue Reading...

Database version management and figuring out which scripts need to be run when deploying the latest version of your web app

By Matt |  Jan 5, 2013  | database, sql, versioning, featured

When you are maintaining multiple web apps across environments it can be difficult to keep track of which scripts need to be run to upgrade the database when it comes time to deploy. If you’re maintaining different versions of web apps across environments when the versions can sometimes be significantly out of sync, the difficulty of determining which update scripts need to be run on deployment can explode.

While there are a ton of approaches to keeping your database under version control, if you want something simple and effective that you can implement with minimal time and effort, consider a DatabaseVersionHistory table in your database.

A database version history table will allow you to see at a glance the state the database is in and by comparing it with the update scripts in source control, you will quickly be able to determine which update scripts need to be run.

Continue Reading...

A JIRA issue tracking FAQ for a small team

This post is intended as a living document that will evolve and grow over time. If there’s something you think I missed or would like something clarified, please feel free to leave a comment.

Who should be reading this FAQ?

Managers, developers, testers, anybody working or contributing on a software project.

This article is generally JIRA specific, however concepts will carry across into other issue trackers such as BugzillaRedmineTeam Foundation Server (TFS) and Trac just fine.

When should a small team be using an issue tracker such as JIRA?

A young startup may initially get away just fine by working informally and by email, and early in the project you’ll want to have as little administrative burden as possible, however as a project and team matures, there will come a time when having a searchable, persistent audit history of business decisions, fixes (and why they were done) and completed tasks will become invaluable.

Continue Reading...

Using Mercurial with a SVN repository in a production environment without any drama

By Matt |  Apr 5, 2012  | mercurial, svn, featured

Why would I want to use Mercurial or any other DVCS client with a Subversion repository?

  • It lets us keep SVN as our central repository
  • Some team members prefer not to use a DVCS for whatever reason so it lets them carry on using SVN without interruption.
  • It allows me to work and commit changes (but not push!), search history and switch between branches completely disconnected. I can continue to work during network outages or while traveling when I don’t have connectivity.
  • You get full, fast history search.
  • Switching between branches is easy and fast.
  • Any automated processes which use SVN (i.e. automated builds and deployments) can continue to operate while everyone moves to DVCS.
  • It’s much easier to perform merges than regular SVN (via export/import patch queues – which I detail later)
Continue Reading...

Wellington Spearfishing

By Matt Button |  Aug 1, 2017  | diving, spearfishing, wellington, new-zealand, featured

A collection of links, with a Wellington focus, which I frequently use to work out if it’s worth going for a dive or not.

Feel free to get in touch if you have any links worth adding.

Webcams

Beacon Hill Webcam

Cornish - Lyall Bay webcam

Victoria University Coastal Ecology Lab - South coast time-lapse camera

Forecasts

MetService - 5 day forecast for Wellington

MetService - 10 day forecast for Wellington

Swellmap - Surf Forecast Conditions and Marine Weather

White Cloud Kiteboarding - Wind report

Windy - Interactive wind forecast maps

Maps

Cawthron Eye - Satellite view

LINZ - Hydrographic raster chart

Water Quality

Shellfish Biotoxin Alerts

Wellington Water Quality

Continue Reading...