Spinn3r is a licensed weblog crawler used by search engines, weblog analytic companies, and generally anyone who needs access to high quality weblog data.
We crawl the entire blogosphere in realtime, remove spam, rank, and classifying blogs, and provide this information to our customers.
Spinn3r is rare in the startup world in that we’re actually profitable. We’ve proven our business model which gives us a significant advantage in future product design and expanding our current customer base and feature set.
We’ve also been smart and haven’t raised a dime of external VC funding which gives us a lot more flexibility in terms how how we want to grow the company moving forward.
In this role you’ll be responsible for maintaining performance and availability of our cluster as well as future architecture design.
You’re going to need to have a high level overview of our architecture but shouldn’t be shy about diving into MySQL and/or Linux internals.
This is a great opportunity for the right candidate. You’re going to be working in a very challenging environment with a lot of fun toys.
You’re also going to be a core member of the team and will be given a great deal of responsibility.
We have a number of unique scalability challenges including high write throughput and massive backend database requirements.
- Maintaining 24 x 7 x 365 operation of our cluster
- Tuning our MySQL/InnoDB database environment
- Maintaining our current crawler operations
- Monitoring application availability and historical performance tracking
- Maintaining our hardware and linux environment
- Maintaining backups, testing failure scenarios, suggesting database changes
- Experience in managing servers in large scale environments
- Advanced understandling of Linux (preferably Debian). You need to grok the kernel, filesystem layout, memory model, swap, tuning, etc.
- Advanced understanding of MySQL including replication and the InnoDB storage engine
- Knowledge of scripting languages (Bash and PHP are desirable)
- Maintaining software configuration within a large cluster of servers.
- Network protocols including HTTP, SSH, and DNS
- BS in Computer Science (or comparable experience)