Please check Part 1 here if you have not already. Part 2: The Payload (Python Jobs) Preparing the Data (files) Disco Distributed Filesystem (DDFS) is a great low-level component of Disco. DDFS is designed with huge data in mind, so it made more sense to use it in my experiment as opposed to any other type of storage, for example, HDFS. Moreover, we can even store job results in DDFS, which we are going to do below....
Prologue This post is my take on reviving an old project (the last commit was 3 years ago) born around 2007/2008 at Nokia Research Center and written in Erlang. What was exciting for me is the fact that Disco project is capable of running Python MapReduce Jobs against an Erlang core, how awesome is that! — Erlang is a synonym for parallel processing and high availability. I successfully built it though and ran a 250M records dataset which is 10GB+ in size using a Python MapReduce job that finished in 28 minutes (improved from 44 minutes) using a cluster of 3 EC2 free-tier t2....
This project is part of the Teensy LC Challenge Teensy Project 001: Blinking LED Checkout the source code on Github. Teensy Setup The environment setup section is common to all the Teensy LC projects, so I created a single post with the required instructions to set up the Arduino IDE, Teensy Loader application, Teensyduino and Linux udev rules. Go to the complete setup guide First Usage: Blinky Program This section is also repeated in the setup guide since it’s basically the first project introduced in this quest....
Introduction Teensy LC (Low Cost) is a 32-bit microcontroller board that you can get from PJRC a company in Oregon, USA, which is owned and managed by Paul Stoffregen. It features an ARM Cortex-M0+ processor designed for low-power, low-cost devices. You can buy it and read more technical details on PJRC’s Teensy LC page. I am writing this tutorial as a complete guide for starting with the Teensy LC development and also as a reference for myself in case I need to revisit it in the future....
Quest 001: Teensy LC Challenge Prologue It all started with a simple tweet. I was going through my parts inventory and found the two items collecting dust — I won them as prizes on HackADay.io for two of my projects there (you can check my profile here). So I won 2 prizes from @hackadayio back in 2015: a "#Teensy LC" & a "Cordwood Puzzle". Thanks @szczys @asiwatch I was busy since then & never used it, so now it's time to put it to Good Use....
Prologue It was a great Friday, the 31st of August 2007 at 22:25:25 I etched my first PCB ever at home, it was the second year of high school and I was so excited about it. After doing some research and information gathering from some friends who happen to be great engineers. This is how I got started with electronics — believe it or not, by building my first PCB using Ferric Chloride FeCl3 solution with 60% concentration and a blank copper Printed Circuit Board (PCB)....