- Examiner: Prices at LDS canneries show inflation for food up between 11 and 49% – “The LDS’s raising food prices at their canneries by 11 to 49% in just three months should be a serious wakeup call to all Americans on the true inflationary conditions that exist in our economy, and that we need to constantly look outside government reports for the true data affecting our spending and finances.“
- Search Engine Watch: More Bounce to the Ounce: Avatar Body Physics Come to Second Life – “Some people are not so thrilled about this new feature. Not because they’re prudes necessarily or have anything against watching things bounce around. But because this feature was created and rolled out at the expense of other problems that need to be fixed on the grid.“
Links for Sunday, March 27, 2010
- Interplanetary Transport Network – “The Interplanetary Transport Network (ITN)[1] is a collection of gravitationally determined pathways through the solar system that require very little energy for an object to follow. The ITN makes particular use of Lagrange points as locations where trajectories through space are redirected using little or no energy. These points have the peculiar property of allowing objects to orbit around them, despite the absence of any material object therein.“
- MSNBC: Science Thrives in Virtual Worlds – “Does the virtual-reality world known as Second Life have anything to offer for real-world scientists? Absolutely — and a trailblazing researcher says the payoffs are sure to increase when the Internet goes 3-D.“
Links for Monday, February 14, 2011
- NASA Science News: Robonaut 2 Set to Launch in February – “R2 will be the first humanoid robot to travel and work in space, so it’ll be training for some big responsibilities.“
- Cory Ondrejka: HTML5 Games 0.1: Speedy Sprites – “With wide adoption and industry support, HTML5 will transform desktop and mobile gaming, creating amazing user experiences that are only a link away. Already, over 125 million people visit Facebook using HTML5 capable browsers just from their mobile phone, and that number skyrockets when we add in desktop browsers.“
- Wayne Chang: From the Richest Man in the World – “I write to you this letter in order to share some of my life experiences, hoping it will contribute to your education, your way of thinking and living, your emotional well-being, your sense of responsibility to yourselves and to others, your maturity, and above all, to your happiness, which should be the result of your daily existence.“
- BoxCar 2D – “Genetic Algorithm Car Evolution Using Box2D Physics.“
- Box2D Physics Engine – “Box2D is an open source physics engine written primarily for games. As the name suggests, Box2D is a purely 2D engine. However, Box2D has grown beyond it’s humble box simulating roots, and can now handle convex polygons and other shapes coming soon.“
PosterBurner is Cool!
I recently designed a poster to commemorate the launch of my AWS book. Starting with a high resolution PDF of the cover, I added images of the first page of each chapter, a picture of the product page on Amazon, a couple of reviews, and a picture of some Argentinian developers each holding a copy of my book.
I designed the poster in Microsoft Publisher and generated a one-page PDF, which I then uploaded to PosterBurner for printing. I paid for the printing and the finished poster arrived a few days later.
Unfortunately, there was a small yet quite visible printing defect — a light pink area on the cover of the book. I messaged the good folks at PosterBurner and they sent me a replacement poster just days later.
The finished, framed poster is now hanging on the wall in my home office as a reminder of the very pleasant time that I spent writing the book in the summer of 2009.
Building Sage (Open Source Math) on Amazon EC2
A quarter or two ago my son Andy took a rather unique course at the University of Washington. In his Math 480b: Programming for the Working Mathematician course, Andy learned about a number of important topics including the Unix command line, Python programming (including classes, exceptions and decorators). In the second half of the quarter they learned about the Sage open source math system.
The course ended by teaching the students how to make a genuine contribution to Sage. They were asked to find an open bug, figure out how to fix it, fix it, and to create and submit a patch. In essence, they learned a very practical skill that is taught all too rarely in school — how to be a contributor to an open source project. This is pretty significant. Despite the presence of the word “open”, I have come to learn that many people don’t understand the actual workings of the process. Walking the students through it, and having them make an actual contribution, will ensure that they leave school with this knowledge under their belt. With any luck it will be easier for them to find jobs and they’ll be more useful and more productive once they start.
Exposing the students to the source code also conveys the message that they have the ability to modify (and hopefully improve) their tools. In the COM 546 class that I am taking this quarter, we discussed the fact that a lot of modern hardware cannot even be opened up (apparently, the newest iPhones are held together using screws that have a proprietary head). While certainly fine if you want to treat your phone as an appliance, this closed model doesn’t encourage customization or hacking. It certainly is not in accord with the Maker’s Bill of Rights.
I watched Andy put Sage through its paces and thought that it looked kind of cool. While I am no mathematician, I could definitely appreciate what it was able to do. Andy pointed me at a blog post written by his professor, and I was intrigued. After finding himself “fundamentally dependent on a closed source non-free program in order to continue my own research,” his professor (William Stein) decided (by coincidence, on the same day that he learned about the GPL) to design the language that would eventually become Sage. The full story is in the aforementioned blog post and is interesting (but far too long to summarize here).
Andy spent about four days getting the Sage code to build on his Acer netbook. At the time I didn’t even realize what a feat this was. For his final project, Andy and his team-mates used Sage to solve something called the Ham Sandwich problem. They documented their work here.
Sage is a very rich and very powerful tool. It includes a large number of powerful packages in the standard distribution, along with a smaller set of optional packages. It has a nice Ajax-powered notebook user interface (you can also access it from a shell prompt). Sage also runs a number of other math packages as child processes and presents a unified interface to them.
Earlier this month I was asked to teach cloud computing workshop at Stanford University. I was told that the students would be from the Applied Math and Computer Science departments. Even though I knew that they used MATLAB, I thought that it would be fun to build Sage and to show them what an open source math tool looked like. Binary versions of Sage are readily available, but I wanted to see what it would take to build it from scratch.
I launched one of the more powerful EC2 instances (a High Memory Quadruple Extra Large) with the 64-bit Amazon Linux AMI. I logged in to the instance and installed a few packages based on the information in the Sage Installation Guide. I installed gcc, gcc-gfortran, gcc-c++, make, m4, perl, tar, readline, and readline-devel.
Then I downloaded the source distribution and captured some pre-build facts. The source tree contained 150 files and consumed about 350 MB of disk space. I also captured the system process count from /proc/stat.
I set the MAKE environment variable to “-j8″ since the EC2 instance had 8 cores and typed “time make.” I watched the build tree run configure again and again, and saw it compiling a ton of code in rapid-fire fashion. The load average never went above 4; there was just not that much parallel building to be done.
75 minutes later, the build was complete. The tree had grown to a staggering 2.5 GB and now contained 84,802 files! Even more interesting, the build process had used 397,484 processes.
I launched Sage from my shell, and typed a command to start the notebook user interface:
sudo ./sage -n port=80 interface='' requirelogin=False openviewer=False
And that was it. I opened up my browser, copied the EC2 instance’s Public DNS name, and pasted it into the address bar, and opened up the administrator page. Here’s what it looked like after I created a couple of notebooks:
I opened up my notebook:
Then I did some simple calculations (my math skills are so rusty it is a wonder that I don’t have Tetanus):
I also drew a cool graph using some of the Sage sample code:
I have this fantasy of actually using Sage to re-learn math, starting from the very basics (Calculus or Algebra II), but there’s no time for that right now.
All in all, I find Sage to be more than impressive. The build process was remarkably clean for such a huge package, especially one that included code from so many different projects.
Just for fun, I also built Sage on an EC2 “small” instance. With MAKE set to “-j2″, this build took 343 minutes, or nearly six hours.
I know that members of the Sage team are interested in building it on EC2 and I hope that they find this post. If you are on the team and would like to pick up where I left off, let’s talk. You can get access to Amazon EC2 through the AWS in Education program.
Links for Thursday, January 27, 2011
- SysAdmin 1138 Expounds: The Linux Boot Process, A Chart – “I did this because Things have Changed from the last time I really studied this. Back when I started it was BIOS, LILO, and Initd. I never did bother to wrap my head around Grub, mostly because the automatic tools have gotten a lot better so knowing it just to install isn’t needed, and I haven’t had the right kind of boot problems where I’d learn it through troubleshooting.“
- Denver Post: From the Archive: Historic NASA Photos – “NASA has partnered with The Commons on Flickr and the Internet Archive to make a collection of 180 historic photos available for public viewing. The photos are arranged into three sections – Building NASA, Launch/Takeoff and NASA Center Namesakes.” – One increment beyond awesome!
- Matt Kruse: Better Facebook – “Better Facebook is a FREE user script that plugs into your browser and adds a lot of great enhancements to your existing Facebook account!“
- Dave Winer: EC2 for Poets, v2 – “I’ve spent the last few months, in the middle of all the other projects I’ve been working on, getting a new version of EC2 for Poets ready. And as of today, it’s reached a certain level of ready-ness. So here goes!“
- Wayne Chang: Building a Sustainable Side Project – “I’m extremely passionate about creating and building new things, so I started a few side projects in early December. A side project, to me, is something that requires less than full time effort on my part but serves a purpose and has high impact in some way. It is an exercise to hone my skills, learn new things, and possibly create a new stream of revenue.“
- seaWALDO – “A bunch (about 104) hand-painted characters that are all similar in size (about 5 inches tall) can be found in the area mapped out (image of playfield to the right) on Capitol Hill in Seattle, Washington. Waldo is hiding somewhere, but after a bunch of hours sniffing the oil fumes while making these lil’ guys, we can say most of them are pretty freakin’ cool.“
- David Trejo: Scraping Made Easy with jQuery and SelectorGadget – “Scraping with jQuery is a real pleasure. Here’s some example code to get you started, it gets the top three articles with their point values from Hacker News. “
Links for Monday, January 17, 2011
- Penelope Trunk: How to Answer the Question, What Do You Do? – “3. Don’t focus on your job. This is not a job interview—it’s an attempt to get to know you so the person can connect with you. So you don’t need to go straight to your job for an answer. Some people have a job that does define them. Some people do not. Once you realize you can go either way on this, you can come up with the best answer for you.“
Finally – Linux on My Desktop
At the beginning of 2011 I finally made a big switch, one long overdue and one that I should have made 15 years ago. Way back in 1985 I was hired into a newly formed group at American Management Systems of Arlington, Virginia. The group was tasked with making Unix easier to use. I started out by writing a program called Directory Shell, an interface to Unix that ran on the so-called “dumb terminals” that were prevalent at the time. The group was eventually spun out into Visix Software and I built another Unix interface, this one graphical and able to run on early graphical window systems such as The X Window System, SunView, and the best-forgotten Environ V.
After leaving Visix in late 1994, I started up my own consulting practice. My first big project was a port of the Looking Glass code base to Linux for a hot startup, itself a spin-out from Novell by the name of Caldera (this was the early and “good” Caldera, not the subsequent “bad” one created through a merger with SCO). The port went really well and I had Looking Glass up and running as planned. Caldera was really ahead of its time when it came to thinking about Linux on and for the desktop and (looking back) I should have paid a lot more attention to what they were doing. Since that time I have always had a Linux server (or two) underneath my desk or in the cloud but only for “server” stuff — like storing data and hosting web sites.
My son Stephen (who is often ahead of the curve when it comes to such things) was an early advocate of open source, Linux, and Ubuntu. For the last couple of years he has openly (and rightfully) ridiculed the last Windows desktop machine, a machine that we had hand-assembled back in 2005. I had promised to give Ubuntu a try as soon as it came time to build another desktop machine.
Last fall, that ancient desktop machine began to fail. First one drive failed in the RAID set. The other drive in the set continued to work for a while, until one day the controller decided that too much time had elapsed since it had synced both drives, and shut everything down. Fortunately, I had backed up most of my data to a newer SATA drive sitting in a USB cradle on my desk. A week before Christmas I was rearranging my home office and moved that old beast into its new location, less than 8 feet from the old one. It refused to power up and it was clear that the power supply (already replaced once) had given up the ghost.
For once, the timing was perfect. Stephen was home from grad school and eagerly spec’ed out a new machine for me (spending Dad’s money is always fun). The finished system included an ASUS P68X58D motherboard, an Intel Core i7 950 processor with a gigantic Noctua heat sink, 6 GB of ECC RAM, a fast 150 GB drive for the OS and a slightly slower 1 TB drive for data, all in a well-ventilated Coolermaster case. I was supposed to order a new NVidia graphics card but forgot to add it to my cart. Fortunately, I was able to extract and reuse the (relatively recent) card from my previous machine.
We bolted it all together and fired it up. As is almost always the case, it worked just fine on first boot. This aspect of the PC industry always amazes me. Random (albeit high quality) PC parts built in factories located all over the world, are almost always compatible.
Stephen had downloaded the newest version of Ubuntu Desktop earlier that day and burned it on to a DVD. He brought it downstairs and we were on our way. We did a complete install and I was up and running in no time flat.
It has now been a couple of weeks since we brought the machine up and I find myself impressed with Ubuntu Desktop. It is fast, smooth, easy to use, and compatible with all of my new and existing hardware.
I installed a proprietary NVidia driver to improve performance, and then installed the latest Second Life client, which worked fine, including the often-finicky audio. I installed Emacs and then used Dropbox to get access to my cloud-based files.
I plugged in my USB SATA cradle and recovered some Firefox bookmarks from my old C drive. This was painless. The system recognized that the drive was formatted with NTFS and mounted it as such. I copied off the files that I hadn’t backed up, including a 3 GB Windows 7 image file (more on that later). Emboldened by this success, I took the drives from my old RAID set and plugged them in to the cradle. The first one was truly dead. The second one (reported as dead by the controller in the old machine) turned out to be working fine. I was able to recover some files that would have been very difficult to obtain otherwise. Three gold stars for Ubuntu.
I listen to lots of podcasts and started to figure out the best way to download them and sync them to my iPod. After trying out gPodder and RhythmBox, I settled on Banshee. Banshee works reasonably well but still has trouble syncing from time to time.
I am still working on sharing my family’s user tree (containing over 70,000 files) and our “camera” tree (another 80,000 files) using Samba. It is very close, but not quite working. I can’t print, either. I installed the proper printer drivers and (per the information here, created an empty directory, but so far no go.
Next on the list was access to Windows. While it would be nice to be a purist, I do need to use Outlook and to edit Word, Excel, and Publisher files almost every day. My first step was to simply open up an rdesktop session to my work laptop. This worked fine but I thought that I could do even better.
I installed the latest version of VirtualBox and then (because I couldn’t find a blank DVD) installed Windows Vista on it. This went smoothly and I installed a trial version of Office 2010 on it. Subsequently, I did buy some blank DVDs, burned that 3 GB Windows 7 image onto one, and installed it under VirtualBox. It is rock-solid and can even see my network shares and my printer. There is one annoying bug which causes the keyboard to stop working from time to time. This is apparently a common problem but no one seems to have a solution. Other than that, VirtualBox works great. I took a snapshot of my virtual machine after installing and authorizing Windows 7, and another after doing the same for Office 2010. I can restore from any snapshot in a few seconds with just a click or two.
Compatibility with USB devices, including my ancient HP scanner, has been excellent. I plugged it in, installed the Simple Scan application, and had no trouble getting it to work.
Ok, so what doesn’t work?
I have not been able to get my Samba shares to work, but this is just a configuration issue. The only Windows application that I really and truly miss and cannot live without is Snagit. I need to take high-quality screen shots for the AWS blog and I have yet to find a way to scale, crop, and create nice torn-paper edges on Ubuntu. I can do the scaling and cropping with GIMP, but the scaling isn’t 1/10th as good as Snagit’s. I can use Imagemagick to create fuzzy edges on my screen shots, but I really like (and cannot seem to duplicate) Snagit’s torn paper effect.
I have not yet tried Skype, and I have not yet tried to attend or to host a webinar.
Net-net, I am very happy with the setup!
Links for Monday, January 3, 2011
- John Wiegley: Git From the Bottom Up – “In my pursuit to understand Git, it’s been helpful for me to understand it from the bottom up—rather than look at it only in terms of its high-level commands. And since Git is so beautifully simple when viewed this way, I thought others might be interested to read what I’ve found, and perhaps avoid the pain I went through finding it.“
- Sergey Melnik: Dremel: Interactive Analysis of WebScale Datasets – “Dremel is a scalable, interactive ad-hoc query system for analysis of read-only nested data. By combining multi-level execution trees and columnar data layout, it is capable of running aggregation queries over trillion-row tables in seconds. The system scales to thousands of CPUs and petabytes of data, and has thousands of users at Google. In this paper, we describe the architecture and implementation of Dremel, and explain how it complements MapReduce-based computing.“
Links for Saturday, January 1, 2011
- Hummingbird: Real Time Web Traffic Visualization – “Hummingbird lets you see how visitors are interacting with your website in real time“
- Official 3DXplorer Blog: 3DXplorer at the University of Washington – “University of Washington is one of the most active universities in the field of Virtual Worlds and one of the rare ones delivering a certificate in Virtual Worlds, under Randy Hinrichs’ leadership.“
- Gephi: Makes Graphs Handy – “Gephi is an interactive visualization and exploration platform for all kinds of networks and complex systems, dynamic and hierarchical graphs.“
- HEAD JS: The Only Script in Your <HEAD> – “Load scripts like images. Use HTML5 and CSS3 safely. Target CSS for different screens, paths, states and browsers. Make it the only script in your HEAD. A concise solution to universal issues.“
- Sourceforge: Ditaa – “A small utility that converts ascii-art diagrams to nice-looking bitmap diagrams automatically. “
- Ditaa: Diagrams Through ASCII Art – “ditaa transforms ASCII art into graphical diagrams. To see it in action, make changes to the text below and click Update the Diagram.“
- Emacs Screencast: Artist Mode – “Artist mode, toggled by M-x artist-mode, lets you draw lines, squares, rectangles and poly-lines, ellipses, and circles with your mouse and/or keyboard. It is extremely useful when inserting text diagrams or figures in your source comments.” – Use this with Ditaa!
- Josh Staiger: Fixing Garbage Characters in the Emacs Shell Buffer – “In many default Linux configurations the stock .bashrc config turns on colorization by default for many shell commands like ls and grep. Commands that colorize their output can cause problems for the Emacs shell buffer.“