Skip to content
- Turning on query logging for PostgreSQL in Amazon RDS 2018-03-09- This is a very specific topic, but I needed to do it to track down performance of a slow web service call. First, go to your Amazon RDS instances tab and select the database that needs logging turned on, then go to “See details” under instance actions. Find the Parameter Group attached to the database. … Continue reading "Turning on query logging for PostgreSQL in Amazon RDS"
- On syncing data 2017-07-17- I’m working on an ETL pipeline, which pulls from a REST API, does some transformations, and upserts it into a database. The REST API occasionally has invalid data (e.g. alphabet characters in the zip code, or misspelled US state names), and also occasionally throws HTTP 500 error pages. I’m supposed to stall the entire pipeline … Continue reading "On syncing data"
- Testing data processing applications 2017-05-08- We’re testing a data processing applications. There are multiple pipeline stages, the boundaries (i.e. inputs and outputs) of which are well defined. The bulk of the code deals with reshaping data from one form to another; there is very little functional logic. Therefore, testing should be focused mainly on how the code reacts to: Null … Continue reading "Testing data processing applications"
- Apple push notifications on Amazon SNS from node.js 2017-04-03- We’re going to hook up three things today: Apple Push Notification Service (APNS), Amazon Simple Notification Service (SNS), and node.js. This has the potential to be a mega-post, so instead, I’m going to write an annotated checklist. Maybe later I’ll turn it into a multi-part series. Overview Let’s talk about how all of these are … Continue reading "Apple push notifications on Amazon SNS from node.js"
- Lumens vs Candelas vs Lux 2017-03-13- Whether you’re growing plants or have an obsession with flashlights, you will encounter the terms candela, lux and lumens. These all refer to measuring light, but what’s the difference? Radiant flux Let’s start from the beginning. Light is composed of photons, which have a specific energy depending on the wavelength. The radiant flux is simply … Continue reading "Lumens vs Candelas vs Lux"