| FreeSA |
| Summary |
Our goal is to have a completely free GNU/Linux operating system installed
on small, inexpensive hardware which allows us to use it not only as a
fully customizable firewall and server appliance, capable of advanced
routing and traffic control, but also as a general purpose computer.
Instead of cramming the entire system into flash, we use the flash-resident
firmware as simply a boot-loader for the full-blown system installed on a
hard disk.
|
| Started | August 2006 |
| Ended | Still active |
| Goals |
- Users can construct the entire system from source, using the latest
stable version of all packages at the time of the build
- Learn and teach others about server appliance possibilities
- Demonstrate how to securely share excess bandwidth
- Document the process so anyone else may learn from it, replicate it,
customize it, or extend it
- De-mystify topics like cross-compilation and toolchain construction
|
| Process & Results |
From the beginning we sketched out
High Level Plan
with sensible (small, achievable) milestones. Since then we have tracked
our progress on our
wiki
and publish our
git repositories.
Success has come from persistence as well as collaboration with others,
including donating a unit to OpenWRT to help them help us.
|
| What & How |
- Install Serial Console
- Flash-less development cycle for rapid change & easy recovery
- learn how to use TFTP to load & boot a kernel
- learn NFS for root & configuring kernel command-line
- Access the hardware
- find linux extensions which support hardware (Thanks, OpenWRT!)
- reverse-engineer OpenWRT boot process and userspace programs to
discover how to enable the internal disk
- Learn how to build a uClibc based toolchain & BusyBox
- collaborate with someone more experienced
- document results and ensure
repeatability
- Built hard disk resident toolchain and system from source
- Create a tiny GNU/Linux system to act as bootloader
- learn BusyBox
- learn how to pivot root in an init script
|
| Team Size | Two. This project is largely the collaboration of one
other developer (Brett Neumeier) and myself, with some additional
contributions. |
| Tools | GNU/Linux, intense toolchain and build work,
shell scripting, some c,
some ruby,
hardware |
| Knowledge Gained |
I knew very little about embedded systems or custom GNU/Linux
distributions. My partner knew more, but there remained a vast knowledge
gulf. Success has come from persistence, searching the web, trial and error, and
help from members of other projects (like OpenWRT) in email and forums.
|
| MySQL Enterprise Monitor |
| Summary |
A couple of years ago I was called in to help out a project with its JDBC
usage. The project was deeply struggling and the code was really hard to
understand. I fixed up the JDBC usage and introduced unit tests. I was
asked to stay on and helped bring that project to a successful release.
We've released regularly, and on time, ever since.
|
| Started | 2006 |
| Ended | present |
| Goals | Provide MySQL DBAs with a tool to help them
identify and correct configuration and tuning issues. Provide developers a
tool to get real insight into how their application interacts with the
database. Create a strong reason for customers to purchase enterprise
subscriptions. |
| Results | The
MySQL Enterprise Monitor, especially
with the introduction of the Query Analyzer, has proven very successful.
Enterprise subscriptions have grown greatly. |
| What & How |
- Introduced (automated) unit testing & metrics for test coverage
- Involve real customers in our planning, go on site to them
- Introduced interfaces
- modularizing code
- testing in isolation
- Introduced refactoring techniques
- rewrite small pieces, not everything at once
- use interfaces for comparing old implementation to new
- built tests first to verify
- use libraries where possible (replace hand-rolled Object
Relational Mapper with Hibernate)
- I advocate for everything I can from "agile" methodologies that makes
sense in our environment and that I believe we can actually adopt our
work-flow to incorporate. For instance, we now have Automated Build,
Test, and Deploy, and the QA team now writes automated acceptance tests,
by contrast, given our distributed environment, we do very little pair
programming.
|
| Team Size | Fourteen. When I joined, there were only four programmers plus a manager, now there are fourteen people including QA and Docs |
| Tools | Java, Tomcat, Eclipse, Hibernate, Spring, Struts, HTML, MySQL, GNU/Linux, OS X, Windows, other *nix |
| Knowledge Gained |
I think it's fair to say that I've mentored 3 or 4 people, and
certainly I've also learned along the way. We learned Hibernate
and Spring frameworks through trial & error and collaboration
with team members as we worked through the docs. |
| Java User Defined Functions for MySQL |
| Summary |
At the 2004 MySQL User's conference, I presented a live demonstration of a
proof of concept for calling Java User Defined Functions from MySQL. The
initial proof of concept was well received by the audience, and within a
year I was teamed up with a MySQL core developer to make something more
real, and ultimately capable of plugging multiple languages and even
performing full-blown stored procedures.
|
| Started | 2004 |
| Ended | 2006, with on-going maintenence |
| Goals |
- Java User Defined Functions plugin for MySQL
- Java Stored Procedures plugin for MySQL
- Stored Procedures in multiple languages for MySQL
- Replace the current Stored Procedure language with plugin
|
| Results | Sadly, business priority changes have put this
project on hold. The most obvious piece of remaining work is to allow the
JDBC driver to make connections to the MySQL server and remain inside the
same transaction. As such, it is still not folded into the main-line
code. Although, what is complete works well and is easily downloaded from MySQL Forge:
External Language Stored Procedures. Antony Curtis and I have
demonstrated the code live at conferences and continue to maintain it in
our own time. |
| What & How |
- Functional proof of concept
- demonstrated calling Java from mysqld was practical
- built a library for calling JNI from MySQL "flavored" C
- demonstrated double-dispatch of C calling Java which calls back to C
- Worked with Antony Curtis to create a specification for a language plugin system.
- re-worked the code to be invoked via the plugin API
- Introduced interfaces to the MySQL Connector/J JDBC driver to abstract the means of communication with the server
- Default implementation uses network
- From within the server, an alternative "Strategy Object" is provided which implements the methods via JNI
|
| Team Size | Two. This project is largely the collaboration of one
other developer (Antony Curtis) and myself, with some additional
contributions. |
| Tools | C, JNI, Java, eclipse, gdb, MySQL, GNU/Linux |
| Knowledge Gained |
Before this project, I had never used JNI. This was also my first
use of gdb (GNU C debugger). Learning about these involved a lot of
trial and error, on-line docs, and asking friends and colleagues for
ideas. |
| Gearman in Java |
| Summary |
The gearmanij
projects aims to create a "pure Java" implementation of the Gearman "client"
and "worker" interfaces to make it easy for Java developers to distribute
work to gearman networks.
|
| Started | 2009 |
| Ended | 2009 |
| Goals |
- Create client interfaces compatible with existing gearman servers
- Create interfaces intuitive for Java developers, not simply port the
C or perl interfaces directly
|
| Process & Results |
First we write some sample code which we belive implements the protocol.
Next we test the examples against the gearmand and see if we get the
expected results. Once we do, we know enough to write unit tests, and
refactor our samples to more solid, modular classes. In a few weeks we
we demonstrated the distributed "reverse" (hello world-like) example.
|
| What & How |
- Rapidly create proof-of-concepts
- Refactor working code into well test units with interfaces
|
| Team Size |
Two main developers with some input from two others so far. |
| Tools | Java, JUnit, Gearman |
| Knowledge Gained |
Gearman protocol |
| MySQL Federated JNI |
| Summary |
MySQL has the ability to query tables which are located
in a remote database. This feature is called "Federated". As implemented,
it can only talk to other MySQL databases, but I demonstrated that we
could use JNI and thus JDBC to talk to the remote database.
Federated JNI
would open the door to writing queries against tables in multiple kinds of
databases.
|
| Started | 2005 |
| Ended | 2006 |
| Goals | Running Proof of Concept |
| Results | The result was certainly exciting, but in the end it
we agreed it was not worth investing in turning it into a part of the core
product. |
| Team Size | This project is largely a solo work with some
collaboration with one other developer (Pat Galbraith). |
| Tools | C, JNI, MySQL |
| Knowledge Gained |
I gained enough insight into the MySQL storage engine structure that
when it was announced that MySQL was going to provide a pluggable storage
engine API, I was able to create the first set of technical documentation
on the subject. These were later improved by Mike Hillyer of the Docs
Team.
|
| MySQL Connector/MXJ |
| Summary |
Make MySQL something that can be easily embedded in Java
applications.
Connector/MXJ
is a thin Java utility package for deploying and managing a MySQL database.
It makes the database appear to be a Java-based component. It does this by
determining what platform the system is running on, selecting the
appropriate binary, and launching the executable. It will also optionally
deploy an initial database, with any specified parameters.
|
| Started | 2004 |
| Ended | 2005 |
| Goals |
- Make MySQL as easy to use for Java developers as Derby
- Support Windows, GNU/Linux, OSX, FreeBSD, Solaris effortlessly for the
Java developer
|
| Results | The product came together quickly and is still in
production. The product has had very few bugs filed against it, and
boasts 95% test coverage |
| Team Size | One. This project was largely a solo effort with
some additional review and collaboration. |
| Knowledge Gained |
The biggest insight here was that we could do all of the platform
specific things without needing to use JNI. With only a little bit of
effort we can determine the platform then execute the appropriate shell
commands for that platform using Java's Runtime.exec() facilities.
|
| Product Sight's "SyncroSpec" Part Specification Manager |
| Summary |
Product Sight was a y2k dot com that embraced Extreme
Programing from day one. I was employee number 5. The market and economy
were turbulent and our goals shifted every few months. Being Agile, we
adjusted constantly. We built a reputation for delivering every time.
|
| Started | 2000 |
| Ended | 2004 |
| Goals |
- Build a Java Web Application for "Product Life Cycle Management"
of highly engineered parts, assemblies, and products
- Deliver additional functionality every two weeks
|
| Process & Results | Although the challenges were many, we
delivered like clockwork. The product we built evolved significantly
over time as requirements changed. Our quality was legendary: we only
ever shipped to clients about 1 real bug per year I was there. We
tracked actual engineering time vs. estimates, and our time estimates
were seldom off by even 10%. The company was acquired about a year after
I left, but I understand the product is still in development.
|
| What & How |
- Delivered every two weeks
- Very high test coverage
- High collaboration & pair programming
- Adaptive to changing requirements
- Developers worked tightly with management with regular feature planning
and estimation
|
| Team Size | Two to seven developers (usually about four), plus
about four other people in sales and management. |
| Tools | Java, Servlets, JSP, Tomcat, VisualAge, Oracle,
MS SqlServer, DB2, GNU/Linux, Windows, Solaris |
| Knowledge Gained |
I was a very novice programmer at the beginning of this project.
Pair programming with masters made a quick journeyman out of me.
I have since given a lecture on Extreme Programming at the
Universiteit van Amsterdam.
|
| Hotpotato |
| Summary |
SETI@home was the first add-hoc grid computing application
I had seen, but it is single purposed.
The idea with Hotpotato was to build a framework for distributing work
across an add-hoc collection of volunteer computers.
|
| Started | 2002 |
| Ended | 2003, with on-going maintence |
| Goals |
- Ensure people can be confident that code won't put them at risk
- Eliminate the need for clients to install job code beforehand
|
| Results | Hotopato is available on
SourceForge.
|
| What & How |
- Security is achieved using Java's security management
- A custom "no privileges" manager is created for running any
untrusted code
- The worst the untrusted code do is chew up CPU and memory
- Created a custom classloader to distribute objects and classes across
the network as needed at runtime
- Use reflection to see the graph of objects needed to execute
the unit of work
- Use the Byte Code Engineering Library to see what additional
class definitions are referenced and required by the classes used
in the object graph
|
| Team Size | One, with some additional contributions from people
I paired with at the Seattle Pair Programming meetups |
| Knowledge Gained |
I learned about Java's security model and the linking model and a lot
about class loaders and class files |
| Seattle Pair Programming Meetup |
| Summary |
I founded a group which got together every Thursday
evening after work in a cafe and we would work on code in pairs.
It was a fun way to get exposed to new languages and tools. After
about 10:pm we'd close the laptops, order beers, and chat about what
everyone was doing.
|
| Started | 2002 |
| Ended | 2005 (I left Seattle) |
| Goals |
- Create a social environment to learn and experiment with code
- Teach and learn new things
- Have fun
|
| Results | We usually had about a dozen people showing up with laptops and tiny projects they were working on. |
| Knowledge Gained |
I was able to teach and learn a lot about Java as well as get some exposure to Scheme, Javascript, python, whatever others were in to. |
| Knuth's The Art of Computer Programming Study Group |
| Summary |
I organized a Knuth Study Group with a couple of friends.
None of us had studied The Art of Computer Programming in school, so we
decided we'd get together every Tuesday, have dinner, and work through
the book at whatever pace we needed to make sure we understood the material.
|
| Started | 2004 |
| Ended | present |
| Goals |
- Read and understand The Art of Computer Programming
- Do all exercises at or below level 20
- Make certain everyone "gets it" before moving on
|
| Results | We move at a slow, but deliberate pace. After about
three years (accounting for breaks) we have completed over 400 pages of text,
and our bookmarks are very near the end of
Volume 1: Fundamental Algorithms. |
| Team Size | There are usually three of us and we all learned
from each other. |
| Tools | We use VoIP to meet virtually since we no longer live
all in the same city ... or even continent. |
| Knowledge Gained | I had no calculus going into this, so some
of the math has been quite a stretch for me. |
| Learning and Teaching how to build Linux from Source |
| Summary |
I discovered that walking through the
Cross Compiled Linux from Scratch
is extremely educational. I have begun to encourage my friends and
colleagues with an interest in learning more about their GNU/Linux system
to walk through the process and I give them a hand when they get stuck.
I've done the process a few times, and that doesn't make one an expert,
by contrast, I can say that I still learn more each time.
But even doing it once, most people
will have a much stronger idea of how their system works, and what the
pieces involved are that make it work. |
| Started | 2005 |
| Ended | present |
| Goals |
- Learn and teach others how to build all software from source
- Gain a better understand of what software is required and why
- Customize the system as desired
|
| Results | Each attempt brings with it more knowledge visible
as more features. My first attempts were not successful. On my third try
I made it to a bash prompt, and eventually all the way to a full Gnome 2D
graphical desktop. My next from scratch laptop build, I hope to learn how
to configure 3D graphics. In early attempts I would try to never deviate
from the book; today, I do many things slightly differently, without fear
because if something goes badly, I have more confidence that I can figure
out a solution. Instead of only asking questions on the
IRC channels or mailing lists now sometimes I can
help others. |