Tutorials, projects, code, and thoughts collected into topic groups I've generously called Book Projects.
Highlights
New Releases
alms_blended.html- ALM v183, Getting Models to Work Together
- redsho, a discrete optimizer for hyperparameter optimization, and a collection of redsho-examples
- ALM v68, Models Start Getting Interesting
Most Visited
- Transformers from scratch
- Setting up an ssh server
- How to convert RGB color images to grayscale
- Convolution in one dimension
Most Loved
I'm most proud of
- Solving an easy reinforcement learning problem on hard mode: Inverting a pendulum
- Naive Cartographer: A Markov Decision Process Learner
- Ziptie: Learning Useful Features
Book Project: Artisanal Language Models
- Chapter 1: Artisanal Language Models
- Chapter 2: Build a custom tokenizer
- Chapter 3: Define a task and write evals
- Chapter 4: Build an end-to-end prototype
- Chapter 5: ALM v2, First-order Markov model
- Chapter 6: ALM v3, Second-order Markov model
- Chapter 7: ALM v6, Sparse second-order Markov model
- Chapter 8: ALM v14, The Search for Data
- Chapter 9: ALM v68, Models Start Getting Interesting
- Chapter 10: ALM v183, Getting Models to Work Together
Book Project: DIY Networking
- Chapter 1: Host your own web server
- Chapter 2: Set up a domain name for your website
- Chapter 3: Make your webserver secure
- Chapter 4: Caring for your webserver
- Chapter 5: Traffic control
- Chapter 6: Automating traffic control
- Chapter 7: Bomb-proof your website with mirroring
- Chapter 8: Moving your code from prototype to production
- Chapter 9: How to start publishing an RSS feed
- Chapter 10: Setting up an ssh server Notes from my home hackery.
- Chapter 11: Writing a bare-bones HTTP client in Python
- Chapter 12: Writing a bare-bones HTTP server in Python
- Appendix A: The word "server" can mean several things
- Appendix B: Addition App, frontend code, backend code
- Appendix C: Graffiti Wall App
Book project: How to Train Your Robot
How to Train Your Robot is a book-in-progress about applied robotics, machine learning, and software engineering.
- Chapter 1: Can't AI Already Do That?
- Chapter 2: Keeping Time with Python
- Chapter 3: Getting Processes to Talk to Each Other
- Chapter 4: Making Animations with Matplotlib
- Chapter 5: Simulating the Physical World
- Chapter 6: Making Your Python Code Run Faster
- [In progress] Chapter 7: Would You Like to Play a Game?
- [In progress] Chapter 8: Deconstructing Sound
- Chapter 9: Ziptie: Learning Useful Features
- [In progress] Chapter 10: Reconstructing Sound
- Chapter 11: Naive Cartographer: A Markov Decision Process Learner
- Chapter 12: Myrtle: A Real-time Reinforcement Learning Workbench
- Chapter 13: A Dead Simple Message Queue
- [In progress] Chapter 14: DIY webserver
- [In progress] Chapter 15: Animations in Javascript
- [In progress] Chapter 16: Testing a distributed system
- [In progress] Chapter 17: BucketTree, an automated discretizer
- Chapter 18: Putting it all together to invert a pendulum
- Chapter 19: Getting started with Arduino
- Chapter 20: Interrogating an Arduino sensor
Book project: Small Tech
Small Tech is a book about doing analytics, data, and machine learning in small and medium-sized bootstrapped companies
Book project: Under the Hood of Machine Learning
- Chapter 1: Choosing between models
- Chapter 2: Separating signal from noise
- Chapter 3: Choosing a loss function
- Chapter 4: Splitting the data
- Chapter 5: Navigating assumptions
- Chapter 6: Optimization methods
- Chapter 7: Optimization with a central tendency model
- Chapter 8: Optimization with a linear model
- Chapter 9: Optimization with a complex model
- Chapter 10: How backpropagation works
- Chapter 11: Decision trees
- Chapter 12: Support vector machines
- Chapter 13: Autocorrelation
- Chapter 14: Convolution in one dimension
- Chapter 15: Convolution in two dimensions
- Chapter 16: Bayesian inference
- Chapter 17: Fully connected neural networks
- Chapter 18: Regularization
- Chapter 19: Softmax
- Chapter 20: Batch normalization
- Chapter 21: What neural networks can learn
- Chapter 22: Convolutional neural networks
- Chapter 23: Convolutional neural networks, in depth
- Chapter 24: Recurrent neural networks and LSTM
- Chapter 25: Transformers
- Chapter 26: Controlling a pendulum with reinforcement learning
- Chapter 27: Approaching human intelligence through robotics
- Chapter 28: Electrocardiogram case study
- Chapter 29: Compressing images from the Mars rover
- Chapter 30: MNIST digits case study
- Chapter 31: CIFAR-10 images case study
- Appendix A: What questions can machine learning answer
Book project: Data Munging
- Chapter 1: What is data
- Chapter 2: How to get good quality data
- Chapter 3: Data types
- Chapter 4: How to use datetime
- Chapter 5: Reading and writing data files
- Chapter 6: Play and record sounds
- Chapter 7: Converting audio between formats
- Chapter 8: Turn images to videos and back
- Chapter 9: How to slice and index pandas DataFrames
- Chapter 10: Create your first database in SQLite
- Chapter 11: Navigating the awkwardness of databases
- Chapter 12: Converting a csv to a database
- Chapter 13: Create your first database in Postgres
- Chapter 14: Data science for beginners
- Chapter 15: How data science works
- Chapter 16: There is more to data science than machine learning
- Chapter 17: Data science archetypes
Book project: Signal Processing Tricks
- Chapter 1: Rate of change
- Chapter 2: How to normalize a signal by mean and variance
- Chapter 3: How to normalize a signal by its minimum and maximum
- Chapter 4: Exponential smoothing
- Chapter 5: 1D convolution
- Chapter 6: 2D convolution
- Chapter 7: How to convert RGB color images to grayscale
- Chapter 8: How to turn a picture into numbers
- Chapter 9: Turn images to videos and back
- Chapter 10: Play and record sounds
Python packages
- buckettree, an automated discretizer
- cottonwood, a teaching-focused neural network framework
- dsmq, a dead simple message queue
- myrtle, a workbench for real-time reinforcement learning
- pacemaker, a steady time keeper for wall-clock sensitive functions
- redsho, a discrete optimizer for hyperparameter optimization
- sqlogging, a SQLite-backed logging package that mimics Python's logging package
- Python packaging in uv, a guide for the bewildered
Algorithms and Methods
- k-nearest neighbors adaptive feature weighting
- k-nearest neightbors data reduction
- k-sparse neural network layer
- Naive Cartographer , a Markov Decision Process learner
- oknn, an online variant of kNN
- Pathfinder, a many-to-many shortest path algorithm
- Redsho, a discrete optimizer (a.k.a Evolutionary Powell's) and a collection of Redsho examples
- Sharpened Cosine Similarity, an alternative to convolution for learning 2D features in neural networks
- Ziptie, an unsupervised feature creation algorithm
Matplotlib Guides
- I. Quick start guide
- II. Three important ideas
- Colors and colormaps
- Layout, background, and multiple plots
- Lines and curves
- Manual animations
- Patches
- Scatterplots and points
- Text, axis labels, and annotation
- Ticks, tick labels, and grids
Making Python Faster
- Make your code run faster
- Multiprocessing for Parallelization
- Multiprocessing for Real-Time Applications
- Threading
- Make your own personal Python toolbox
Unsolicited Advice
- Being a Staff+ Data Scientist in 2026
- Build a strong distributed data science team
- Choose your professional path
- Decide and commit
- Get to know your new company
- How to be an influencer
- How to choose a project
- How to choose your tools
- How to get a job you like
- How to get hired as a data scientist
- How to solve a hard problem
- Imposter syndrome
- On microsuffering
- Oversimplify your communication
- What I learned running online courses
- What to do when a leader does something wrong
Resources
- Calculus
- C++
- Git
- Linear Algebra
- Machine learning
- Mental Focus
- NumPy
- Python
- SQL
- Statistics
- Useful bash/zsh tricks
- uv cheatsheet for pip users
- Bot tarpit (Read more)
Everything Else
- A love letter to Empirical Design in Reinforcement Learning
- Handling stochastic non-integer delay in reinforcement learning
- Customizing syntax highlighting in vim on markdown files
- Data Revolution Podcast
- Machine Learning Street Talk ML from nuts and bolts
- TRIPLE BAM!!! Podcast
- What I learned building an online school