Monthly Archives: November 2012

Getting Juju With It

At the UDS in Copenhagen I finally had time to attend a session on Juju Charms. I knew the theory of Juju, which is that allows you to easily deploy and link services on public clouds, locally, or even on bare metal, but I never had time to try it out. The Charm School (registration required) session in Copenhagen really showed me the power of what Juju can give you. For example, when I first setup my blog, I had to find a webhost, get an an ssh account, download WordPress, install it, and dependencies, setup mysql, configure WordPress, debug why they weren’t communicating, etc. It was super annoying and took way too long. Now, imagine you want to setup ten blogs, or ten instances of couchdb, or one hundred, or one thousand, and it quickly becomes untenable.  With juju, setting up a blog is as simple as:

  • juju deploy wordpress
  • juju deploy mysql
  • juju add-relation wordpress mysql
  • juju expose wordpress

A few minutes later, and I have a functioning WordPress install. For more complex setups and installs Juju helps to manage the relationships between charms and sends events that the charms react to. This makes it easy to add and remove services like haproxy and memcached to an existing webapp. This interaction between charms implies that the more charms that are available the more useful they all become; the network effect applies to charms!

So after I got home, Charm School had left me energized and ready to write a charm, but I didn’t have any great ideas, until I remembered an app that I’ve used before called Tracks. Tracks is a GTD app, in other words, a fancy todo list. I’d used it hosted before, but my free host went offline and I lost all my to do items. Hosting my own would be much safer. So I started working on a Tracks charm.

If you need an idea for a charm, think about what tools you use that you have to setup, what software have you installed and configured recently? If you need an idea and nothing stands out, you can check out the list of “Charm Needed” bugs. Actually you should check that list regardless to make sure nobody else is already writing the same one.

With an idea in hand, I sat down to write my Charm. Step one is the documentation, most of which was contained on this page “Writing a Charm“. I fully expected to spend three weeks learning a new programming language with arcane black magic commands, but I was pleasantly surprised to learn that you can write a charm in any language you want. Most charms seem to be shell scripts or Python and my charm was simple enough that I wrote it in bash.

During the process of charm writing you may have some questions, and there’s plenty of help to be had. First, the examples that are contained in the juju trunk are OLD and I wouldn’t recommend you follow them. They are missing things like README files and don’t expose http interfaces, which was requested for my charm. Instead I’d recommend you pull the wordpress, mysql, and drupal charms from the charm store. If the examples aren’t enough, you can always ask in #juju on freenode or use askubuntu.com. Once your charm works, you can submit it for review. You’ll probably learn a lot during the review, every person I’ve talked to has.

Finally after a bit of work off and on, my charm was done! I submitted it for review, made a few fixes and it made it into the store.

I can now have a Tracks instance up and running in just a few minutes

I’ve barely scratched the surface here with this post, but I hope someone will be energized to go investigate charms and write one. Charms do not use black magic and you don’t need to learn a new language to write one. Help is available if you need it and we’d love to have your contributions.
If you go write a charm please comment here and let me know!
Tagged , ,

Nexus7: 32Gb Nexus7 with 3g Has Installation Problems, Will Be Fixed Soon

This is a a brief post to alert everyone that the 32Gb Nexus7 with 3g cannot currently install Ubuntu.  The problems have been fixed, but we’re not going to re-roll a Quantal based image for this. You’ll get a fix when the Raring builds are ready, which should be soon, hopefully this week. The issue, if you’re curious, is that the new radio changed the device ID where we write the rootfs into. The new code will not make assumptions about the layout and will determine it dynamically.

Tagged , ,

Nexus7: What Kind of Bugs Are We Seeing the Most?

I was asked last Friday about what type of bugs we see the most on Nexus7.  Right now we have about 75 unfixed bugs, and from having walked through that bugs list about twenty times, I know that we can break the bugs down into some categories. These are arbitrary categories and subject to debate, but they show the patterns I see in the bug list:

Kernel/Kernel Config/Drivers: 9 bugs

The initial kernel we used was the Android 3.1 kernel, which included binary drivers. This kernel has configuration and code differences from the standard Ubuntu kernel. The kernel team at Canonical is working on trying to get the kernel we’re using as close to Ubuntu as possible. This includes things as simple as enabling more modules and as complex as merging in patches so we can support things like overlayfs.

Onboard Related Issues: 9 bugs

For mobility impaired users, Onboard is a part of daily life. For most of us, we only have to use it when we’re playing with a tablet device, so I think it’s great that these bugs are getting some attention. One bug that was recently fixed by marmuta that should lead to Onboard launching 7x faster with certain themes, including the default theme. Some of the bugs in this category are not issues with Onboard itself, but impact Onboard specifically.

Unity/Nux: 6 bugs

Unity and nux have some bugs that impair the usability of the device and a couple bugs that lead to crashes or lock-ups. This is the area I know the least about. Many of the bugs here are sitting in the upstream project as “New”, if you can help confirm them upstream or even find an older dupe, please do.

Tegra3/nVidia: 6 bugs

We found several bugs that seem to be Tegra3 related, Tegra drivers related, or bugs that we need some input from nVidia on, for example, the sound only works after suspend/resume issue.  Not all of these may really be tegra related issues in the end, many require more investigation.

—–

So these are my top four categories, but that still leaves over half the bugs out. There are some smaller categories, which I’d list as Touch, Performance, and Bluetooth, and then there is Misc aka Everything Else.

Categories aside, another interesting thing I’ve noticed that aside from bugs that are specific to the kernel, drivers, or chipset, almost all the bugs we’ve found were confirmed on other platforms; usually they’re confirmed on a someone’s amd64/i386 laptop. Finding, bringing attention to, and fixing these bugs means shows that we’re achieving one of our goals, which is to fix issues in Ubuntu Core. These fixes will benefit all platforms. There are only a small number of bugs that we have not been able to confirm on other platforms yet, can any of my readers do so?  Here are the ones that stand-out in my mind:

Also I’d like to thank all the new contributors we’ve had in the past couple of weeks, we’re glad for all your help on bugs in any form you can provide it.

Below are the full bug lists for my generated categories:

Kernel:

  • 1068672 webcam support
  • 1072320 please consider adding OTG charging support to kernel
  • 1075549 please include fw_bcmdhd.bin and bcm4330.hcd in linux-firmware for support of the nexus7
  • 1076317 overlayfs support
  • 1070770 bluetoothd dies with glibc malloc memory corruption when used with brcm_patchram
  • 1071259 Setting brightness all the way down actually switches off the display completely
  • 1073499 please consider turning on all possible modules for external USB devices
  • 1073840 Sync kernel configuration with the one from the Ubuntu kernel
  • 1074673 JACK server fails to start

OnBoard:

  • 960537 Dash search box doesn’t unhide Onboard on-screen keyboard
  • 1078554 Onboard doesn’t respect launcher icon size
  • 1081227 onboard should optionally stay hidden if a keyboard is present
  • 1075326 On screen keyboard doesn’t re-position in order to see input
  • 421660 gksu’s and gksudo’s modal password prompt prevents OnBoard’s virtual keyboard input, causing accessibility issues
  • 1079591 onboard can be made thin to the point of unusable
  • 1071508 Onboard onscreen keyboard isn’t always shown when text input selected
  • 1077260 When using software center search, onboard goes away until text box is reslected after entering 2 chars
  • 1077277 Keyboard can’t type into Firefox bookmark dialog

Unity/Nux:

  • 1065638 Unity panels don’t display visuals
  • 1072249 Using desktop switcher via touchscreen causes Unity launcher to stop working
  • 1045256 Dash – It should be possible to vertically scroll the Dash left clicking and dragging
  • 1055949 Unity panel shadow appears as solid black bar on GLES/ARM (Pandaboard, Nexus 7)
  • 1075417 Unity panel/launcher width don’t scale with system DPI/font settings
  • 1070374 unity cannot be cleanly restarted from the command-line on Nexus7

Tegra3/nVidia:

  • 1065644 plymouth causes a hard reset of the nexus
  • 1068804 sound only works after suspend/resume cycle
  • 1070283 after reboot, framebuffer of previous boot appears on screen
  • 1073096 Screen is corrupted between rotations
  • 1067954 control-alt-f1 to bring up a VT shows a blank (black) screen
  • 1070755 screen rotates to portrait sometimes

PS – Thanks to Chris Wayne for vetting my bug category list.

Tagged , , ,

Nexus7: You can help with bugs, even if you don’t have a device

There are dozens of ways that the rest of the community can participate in the Nexus7 project, but I’d like to call out one facet that I’ve been working hard on since I got back from Copenhagen. Chris Wayne and I spend a large portion of our day going through the Nexus7 bug list doing bug triage. In brief, what we do is:

  1. If the bug is unclear, ask the submitter for more info in a comment
  2. Checking for duplicate bug reports inside the Nexus7 project
  3. Confirm that the bug really happens on the Nexus7 device
  4. See if the bug occurs on our laptops/dev boxes (x86 running Quantal)
  5. Search the upstream launchpad projects to see if the bug is known already, and mark the Nexus7 one as a duplicate if so
  6. Search gnome bug reports to see if we can link to one
  7. File upstream LP and gnome bug reports if none exist
  8. Testing fixes from upstream
  9. Set bug priority
All of these are standard Ubuntu Bug Triage tasks that anyone from the community can do. Better yet, all of these except for #3 and maybe #8 can be done without even owning a Nexus7.

So today, I’m officially asking the community to help, if you’re interested in helping with the Nexus7 work and don’t know where to start, bug triage is a great place and we could use your help.

So how do you start?  A good place to start is by reviewing the New bug queue and then following the Triaging guide and try to get enough information so that a developer can get started on it. You should also join the #ubuntu-arm channel on freenode, where we can discuss bug status and priority. Feel free to ping me (mfisch) or Chris (cwayne) if you have questions about a bug or how to help.

You may also consider joining the Ubuntu Bug Squad, and later Ubuntu Bug Control if you want to keep doing this work generally in Ubuntu.

On a personal note: when I first started working on Ubuntu, I joined the Ubuntu Bug Squad because there’s really no better way to learn about the components of Ubuntu than to dive into a bunch of bug reports. After a while doing Bug triage work, I was very comfortable with launchpad and bug triage procedures. I also had a better feel for how the components of the system worked together, it’s amazing how much you can learn from digging into bug reports!

Tagged , ,

New Nexus7 Image Available – Check Out the Change List Before Installing!

A new Nexus7 image just posted and can installed using the standard installer and install process. However before you rush out and re-install you should note the changes, only one of which requires you to do a full re-install.

The only fix you need to really reinstall for is this hostname fix:

The other changes are all in the kernel. These changes below can be installed by doing sudo apt-get update && sudo apt-get install linux-nexus7. This will upgrade you to version 3.1.10-7.11 of the kernel:

  • Enable ISO support
  • Enable NFS support
  • Add battery information (upower –dump now works)
  • Enable LXC support
  • Enable SND_USB_AUDIO, disable SND_HDA_INTEL

Over the next few weeks you should expect more changes to come through the standard apt-get upgrade process. These will include syncing the kernel config with the standard Ubuntu one and bug fixes in non-kernel packages.
We may not release a new image again until we have nightlies working. Please note that you should NOT enable any of the standard quantal archives and upgrade things from there. That will supercede some of our fixes like the ones in Nux and you will likely end up with an unusable system.

Tagged , ,

Hacking and Rebuilding the Ubuntu Nexus7 Image

WARNING: This process has changed since the switch to Raring. I don’t yet have a new update for the process.

EDIT: I just added some notes about extracting files from the boot.img in the post.

Several people asked me some questions last week abut how the Nexus7 image is built and how they can hack it. Hopefully this post will help to answer some of those questions. Note that nothing described here is supported, it is just presented here to enable people interested in hacking the image to get going. This process requires the tools simg2img and make_ext4fs which you can download pre-compiled binaries for from here.

Hacking a pre-built image rootfs.img

    1. Take the rootfs.img file as input and use the tool simg2img to unpack it. It will be a large file when unpacked, 28G for the 32GB tablet, 13G for the 16GB tablet, and 6G for the 8GB tablet.

mfisch@caprica:~/build$ ./simg2img rootfs.img rootfs.ext4

    1. Mount the rootfs.ext4 file

mfisch@caprica:~/build$ sudo mount -o loop rootfs.ext4 tmpmnt/

    1. Inside tmpmnt, you’ll find the original rootfs.tar.gz. Copy this file out and unmount the directory. You can also remove the rootfs.ext4 file.


mfisch@caprica:~/build/tmpmnt$ cp rootfs.tar.gz ..
mfisch@caprica:~/build/tmpmnt$ cd ..
mfisch@caprica:~/build$ sudo umount tmpmnt/
mfisch@caprica:~/build$ rm rootfs.ext4

    1. Extract the rootfs.tar.gz


mfisch@caprica:~/build$ tar -xvzf rootfs.tar.gz

    1. The extracted filesystem is in ./binary/casper/filesystem.dir. You can copy files into and out of here or modify files.

Once you’re done with the changes, you need to rebuild the rootfs.img file. The first step is to re-tar and recompress the unpacked files.

mfisch@caprica:~/build$ tar -cvzf rebuilt.tar.gz binary/

From this point you can use the same process we use to build images and then flash them, following the process below.

Building or rebuilding a rootfs.img file

Given a tarball image, our image building script basically does a couple things:

  1. Extract the kernel and initrd from the rootfs.tar.gz
  2. Write a bootimg.cfg file out using the right values for the Nexus 7.
  3. Create a boot.img file using abootimg using the inputs of the kernel, the initrd, the bootimg.cfg. Note: We had to do some work here to make sure that the initrd was small. I think the limit was 2MB.
  4. Take the rootfs.tar.gz, and using the tool make_ext4fs, create a sparse ext4 filesystem and call the output rootfs.img.

We wrote a script to do all this which makes life easier. This process may change as we implement these image builds on cdimage.ubuntu.com and this script may not be updated, but it should be enough to get people hacking. If you do anything cool with this or have fixes for Ubuntu, please let me know or send a patch to one of our bugs.

Hacking/Rebuilding boot.img

After a few questions I decided to add a brief note about how to hack and rebuild the boot.img. It’s pretty simple and uses the abootimg tool which is in universe for quantal.

To extract the files, use abootimg -x

mfisch@caprica:~/upload-scripts$ abootimg -x boot.img
writing boot image config in bootimg.cfg
extracting kernel in zImage
extracting ramdisk in initrd.img

To rebuild, you see see the script I referenced above, or just run abootimg –create:

mfisch@caprica:~/upload-scripts$ abootimg --create newboot.img -k zImage -f ./bootimg.cfg -r initrd.img
reading config file ./bootimg.cfg
reading kernel from zImage
reading ramdisk from initrd.img
Writing Boot Image newboot.img

Tagged , ,

12 Months at Canonical

Last fall, I gave notice at my old job, my last day was to be October 28, and on November first, I started at Canonical.

My first day at Canonical was interesting, it began early with a drive to the airport and a flight to Orlando for UDS-P. It was a great way to start a new job, I left the early 10″ (250mm) Colorado snow behind for palm trees and meeting my new team. I met most of them while enjoying dinner and beers in the evening breeze of Orlando by the pool at the Caribe Royale.

I had to shovel my deck so I could grill when I had some of my family visit

My wife, left behind in Colorado with my parents and our kid who had just contracted Chicken Pox was less than pleased with these photos taken by the pool at the Caribe Royale

Once I returned to Colorado, I soon discovered my job at Canonical was one of a radical generalist. There had not been a single week that went by where I didn’t do something new or work on something new. Our team’s mission is to take Ubuntu and make it work great for customers on various ARM-based platforms. “Make it work great” means that almost anything you’ve ever used in Ubuntu, we’ve had to fix or tweak during one of our projects. In addition to hardware, we’ve also been able to work on some cool features, like remote login and improving test automation in checkbox, and too many more to mention here.

It’s been a fun, educational, and challenging twelve months and I hope the next twelve continue the trend.

Tagged ,