Category Archives: Ubuntu

ubuntu-bug is for lightdm, Soon!

A bug triager’s best friend is ubuntu-bug. Special per-package hooks in /usr/share/apport/package-hooks collect logs and config files that make bug triage and analysis much easier. The scripts also help to avoid the “Can you please post the … file?” back and forth that tends to waste time and many time ends up in an incomplete bug being marked expired. (As an aside when a bug submitter is asked for more info, the bug is marked Incomplete until it is updated, if it’s not updated in 60 days, it expires).

What does this have to do with lightdm? So tonight I shared my lightdm blog post with some people and Bryce Harrington suggested that I write an apport script for lightdm. I mentioned this brilliant plan to Robert Ancell and it turns out that lightdm has one already! Well my work is done!  Except…  The existing script wasn’t being packaged or installed, was collecting logs with the wrong (old) names, and didn’t collect the configs. All of that is fixed tonight with a MP I made, based on the code that Xorg’s apport script uses. Once it’s merged and released (probably not until R opens) we will have an apport script for lightdm. Once that happens, you can file a bug by simply typing:

ubuntu-bug lightdm

There’s no apport script for any of the the greeters, at least not that I know of for unity-greeter or lightdm-gtk-greeter, but they’d need the same data. The only thing that this script doesn’t do is get a screencap/picture, that’s up to you guys.

Tagged , , ,

Lightdm Bugs: What You (the Bug Filer) Should Know

I’ve been spending some time off and on cleaning and working on up the bugs in the lightdm project. Unfortunately someone who takes on this task, lightdm ends up with a lot of bugs that are not necessarily lightdm bugs and also lots of bugs with incomplete information. In a perfect world, everyone who was going to file a lightdm bug would read this, for now I’ll settle for spreading this information to anyone who reads my blog.

First a bit about the architecture. The job of the display manager (prior to lightdm) was to be a daemon, manage logins and sessions, run the display server, and provide a GUI to interact withe the user. Unfortunately since everyone has a favorite GUI technology, we ended up with a ton of different Display Managers (GDM, XDM, KDM, etc). The idea with lightdm was to separate the back-end work from the GUI. Everyone could re-use the back-end (lighdtm) and write their own GUI, also called a greeter, on top. So that’s where we are now, lightdm is a daemon process that manages everything, and a greeter, usually unity-greeter for Ubuntu or lightdm-gtk-greeter for Xubuntu deals with the user interaction. A PDF presentation on the architecture from the Desktop Summit is available here. Note: I assume Kubuntu uses the lightdm-qt-greeter but I’m not sure, if you know please comment here.

So why should you care about the architecture just to file a bug? The reason is that if you get your bug to the right package it will speed up the response. So here’s a real basic decision tree for you to follow when deciding what package to file against:

  1. Is the issue with something you’re seeing on the screen?  If so, it’s probably a greeter bug. If you’re using default Ubuntu, file against unity-greeter. If you’re using Xubuntu file against lightdm-gtk-greeter. Special case: Did the greeter load but the graphics look a little screwed up?  That could be an Xorg driver issue.
  2. If no greeter started, if you started in “low graphics mode”, or it doesn’t fit with case number 1 above, file it against lightdm and we can move it later if we need to.
For all bugs you will also get a much better response if you provide full and correct information from the beginning. This means providing the following details:
  • Like all Ubuntu bugs, you need to include steps to reproduce.
  • For all bugs we need the version of ligthdm you are using and the name of and the version of the greeter you are using. If you don’t know which greeter you are using, you can look at /etc/lightdm/lightdm.conf and check out the line that contains greeter-session.
  • If you have a greeter problem, for example, some of the graphical elements don’t look correct or you have problems entering your username, please include a screen-cap.
  • If you’ve messed with the standard lightdm config file, please attach /etc/lightdm/lightdm.conf to the bug.
  • For all greeter and lightdm bugs it helps us to include the contents of /var/log/lightdm.  You can tar them all up and post them. The logs will include your username, if you don’t want this to be known, then edit the logs and replace it with something else. The logs require root to read, so here’s a gather snippet you can copy:
sudo tar -cvzf ~/lightdm_logs.tgz /var/log/lightdm/*

If you plan on filing a bug against lightdm or a greeter following these steps can really help us out and hopefully get you a resolution quicker.  Thanks!

Tagged , , ,

Becoming a MOTU

Ubuntu relies on “Masters of the Universe” or MOTUs to keep the universe and multiverse components of Ubuntu in good working order. Unlike main, which is officially supported by Canonical, universe and multiverse are community supported. So MOTUs are community members who spend their time adding, maintaining, and supporting these repos.

Even though I work for Canonical, I don’t have any special powers. If I want to upload a package to universe or multiverse, I need to talk to a MOTU or become one. In order to become one, I need to do the work and gain the experience required, just like anyone else in the community would. Since I came into this job without having done much of this type of work before, I’ve started the process of becoming a MOTU by working with a mentor, Robert Ancell. Robert has helped me with the process, pointed out mistakes, and suggested packages to work on and has been a big help along the way.

Note: You must also be a Debian developer to get the battle cat. Also, Robert Ancell does not have blond hair.

So why do I want to become a MOTU? Personally, my motivation has several aspects:

  • I want to give back to the community
  • I want to learn more about Ubuntu and about the processes being used
  • It will make my job easier to have upload rights, more community contacts, and more experience with MOTU processes

I started this process last week by doing some gnome 3.6.0 updates, these were fairly simple, but helped me learn the process and tools. Since then, I’ve done five gnome 3.6.0 updates and three other non-gnome updates (two of which will be waiting until post-quantal to be uploaded).

As I progress in this goal, I’ll make updates on my blog and given an overview of some of the tools and processes being used.

If you want to start along the MOTU track there is no better place to start than the MOTU wiki and no better time than when R opens for development in November. While you wait for R to open, I’d recommend you read up on the policies and procedures and maybe make a dry-run through a package that can be updated in R. If you need help #ubuntu-motu on freenode is the place to ask! If you’ve already started, ping me on IRC (mfisch) or leave a comment here, I’d love to have another MOTU candidate to discuss things with and would be happy to assist you as well.

Tagged , ,

Web Cam Resolution Revisited

Yesterday I was playing around with determining webcam resolution by reverse engineering how fswebcam worked, using the ioctl VIDIOC_TRY_FMT. I keep working that evening on my code and was thinking of filing a bug against v4l2-ctl or the driver until I made a discovery: having fswebcam take a picture at a specified resolution actually changes what v4l2-ctl returns, here’s an example that’s been truncated to keep it short:

mfisch@caprica:/tmp/luvcview-0.2.6$ v4l2-ctl -V
Width/Height : 1280/720

mfisch@caprica:/tmp/luvcview-0.2.6$ fswebcam -d /dev/video0 -r640x480 /tmp/foo.jpg
Writing JPEG image to '/tmp/foo.jpg'.

mfisch@caprica:/tmp/luvcview-0.2.6$ v4l2-ctl -V
Width/Height : 640/480

Hey look, v4l2-ctl changed it’s answer! So is this a bug? I dug into the v4l2 docs and it turns out that you can make it change it’s answer again by picking a different resolution with fswebcam. This has to do with how fswebcam works.

fswebcam first calls VIDIOC_TRY_FMT and that ioctl returns and sets some parameters on the image. For example, if you request an image that’s 10000×10000 it will return back a success, but set the width and height to something like 1280×720 (depending on your hardware).

Next fswebcam calls VIDIOC_S_FMT. Unlike VIDIOC_TRY_FMT, S_FMT actually sets what the driver’s format is. So, if you take a picture at 640×480, the driver will keep that internal format as default until you call it again with a larger value. This is all described in the v4l2 spec.

So, I was wrong, the driver is fine and is in fact behaving as the spec requires. So given this information, I figured that to find the true max, I could just start with a huge value, like 10000×10000 and see what value TRY_FMT came back with. That’s where I left it last night, until a colleague, Josh Poulson, pointed me to luvcview.

luvcview can list all the supported formats and even the time intervals between frames, it prints a long list, so here’s a truncated version:

Device information:
Device path: /dev/video0
{ pixelformat = 'YUYV', description = 'YUV 4:2:2 (YUYV)' }
{ discrete: width = 640, height = 480 }
Time interval between frame: 1/30, 1/20, 1/15, 1/10, 1/5,
{ discrete: width = 640, height = 400 }
Time interval between frame: 1/30, 1/20, 1/15, 1/10, 1/5,
{ discrete: width = 352, height = 288 }
Time interval between frame: 1/30, 1/20, 1/15, 1/10, 1/5,
....

And that’s what I needed. It turns out that there’s an experimental ioctl called VIDIOC_ENUM_FRAMESIZES that luvcview is using. This information highlights the good and bad parts of the web. It was easy for me to take existing tools and dive into the problem I was trying to solve with virtually no knowledge of v4l2, but I made some bad assumptions. But it’s also thanks to the web, and my partially incorrect blog post that I know so much more now.

Tagged , , , ,

Determining Web Cam Resolution (Updated)

After you read this, please read my follow-up post here.

Update the bottom

I spent some time today trying to determine the answer to a seemingly simple question, “What resolution is my web cam?” There are a myriad of tools that display webcam info including v4l-info, v4l2-ctl, xawtv, fswebcam, etc. v4l-info and v4l2-ctl both claimed 1280×720, but I had some issues with the images before, so I wasn’t sure I trusted those values. So I decided to use fswebcam to take some stills. I was able to take 640×480 pictures with fswebcam just fine, so I tried 800×600, fswebcam balked at this value:


mfisch@caprica:~$ fswebcam -d /dev/video0 -r800x600 /tmp/foo.jpg

--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Adjusting resolution from 800x600 to 640x480.
--- Capturing frame...
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to '/tmp/foo.jpg'.

Note that fswebcam adjusted the resolution down to 640×480. Maybe that’s my max resolution? That doesn’t match what the other tools report so I tried again with 1280×720:


mfisch@caprica:~$ fswebcam -d /dev/video0 -r1280x720 /tmp/foo.jpg
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to '/tmp/foo.jpg'.

Note that it didn’t adjust the resolution this time. Let’s go for the limit, maybe it can do 10000p!


Writing JPEG image to '/tmp/foo.jpg'.
mfisch@caprica:~$ fswebcam -d /dev/video0 -r10000x10000 /tmp/foo.jpg
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Adjusting resolution from 10000x10000 to 1280x720.
--- Capturing frame...
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to '/tmp/foo.jpg'.

Nope, sadly it adjusted down again to 1280×720, so no 10000p for me. I dug into the code inside fswebcam to see how this works and it all boils down to an ioctl() called VIDIOC_TRY_FMT. What this ioctl does essentially is say “hey, can you give me this resolution and this format”? If it returns a non-negative that means “yes, I support that format”, and the driver modifies the fmt structure to report back what resolution it can support, generally (perhaps always?) adjusting the size downwards. You can see this being used in the file src_v4l2.c file in the fswebcam package. So when I requested a 10000×10000 image captured in MJPEG (the default) it downsized my request to the nearest supported size, 1280×720.

Note: VIDIOC_TRY_FMT is explained in detail in this lwn article.

1280×720 matches what v4l2-ctl -V showed me here:

mfisch@caprica:~$ v4l2-ctl -V
Format Video Capture:
Width/Height : 1280/720
Pixel Format : 'MJPG'
Field : None
Bytes per Line: 0
Size Image : 1843789
Colorspace : SRGB

What is more interesting is that I had a co-worker run this same set of tests and his results didn’t match. v4l2-ctl claimed his max was 640×480, but fswebcam’s calls to VIDIOC_TRY_FMT would allow 1280×1024, so his driver was caught in a lie!

My co-worker uses a System76 Lemur Ultra which has a 1.3MP camera. 1280×1024 = 1.3MB, so I think that it is taking pictures that size and not upscaling them (which was my earlier thought).

Given this info, I’m still planning on using v4l2-ctl -V, but I think if you want to be sure you should call VIDIOC_TRY_FMT and using fswebcam is a simple way to do so.

Update:

It seems that sometimes my driver is wrong as well. After taking a bunch of still images tonight during testing, look what mine now reports:

mfisch@caprica:~/qa/checkbox/scripts$ v4l2-ctl -V
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'MJPG'
Field : None
Bytes per Line: 0
Size Image : 614400
Colorspace : SRGB

There is some kernel bug at work here, if I figure out more, I’ll file a bug and post an update here.

Tagged , , , ,

Coders, 3 Year Olds, and Unit Tests

Anyone who has kids knows that three-year olds are starting to learn that the world is full of complex rules. Don’t go on red. Look before you cross the street. You can’t just watch TV all day. What every three year old wants to know is “why?”. I answer “why?” to my kid about twenty times a day. I’ve found that from all the teams I’ve worked on, coders are the same. Tell a coder that they need to fill out a new bug report form or write unit tests and they will ask “why”. Now, like a three year old you can threaten a coder (usually with a bad performance review rather than a timeout), you can bribe a coder (maybe a bonus or award instead of candy, or maybe just candy), or you can convince that it’s in their best interest to it. Like three year olds, it’s best to convince coders that doing something is in their best interest rather than threatening or bribing.

When it’s time to discuss unit testing, what most managers do is bring out some slides about quality and fill them with buzzwords about “quality metrics” or even worse, they bring up some industry standard that promises lots of paper work like CMM or ISO9000. I’ve got to be honest, while almost every coder I know cares about the product and the customer experience, most I know could care less about something like CMM or ISO9000.

So what is the right way to make coders care about unit tests? From my personal experience, unit tests give me four key benefits:

  1. Prove that the API is not broken
  2. Increase speed of execution
  3. Catch issues before they get shipped
  4. Allow me to really understand an API/project

Prove That the API is Not Broken

I really like writing unit tests that test an external API. I can use my tests to ensure that the API is staying stable. This is important not just to catch places where someone adds an argument to a function, but when someone makes a change that modifies the behavior of the API. Once an API ships, people start relying on the behavior of it, whether it was specified or not (wrongly or not). For example, some accomplishments-daemon code will throw a KeyError if you pass in an invalid key to a dictionary. So regardless as to whether this was specified in the documentation, someone may have code that relies on that, and so I test it. (I know that relying on implicit, undocumented behavior of an API is bad, but almost everyone does it.) If the code is changed next week to catch the KeyError instead and return None, the unit tests will at least tell us and we can decide whether at the very least the API version needs to roll.

Increase Speed of Execution

This is my favorite reason for writing unit tests. Well written tests let you isolate testing of one function all by itself and make it easier to debug an algorithm without all the setup and teardown. In a previous job I was writing software to get show listings from a TiVo and download shows. I could not make much progress unless I was in my office on the same subnet with my three TiVo devices. Also these devices were sometimes taken by the test group or marketing for various reasons, delaying my work. So what I ended up doing was writing unit tests and making a fake TiVo device, a mock, that behaved exactly like a real device (from an API stand-point). This not only allowed me to work anywhere, it let me skip the tedious setup, device discovery, and remote debugging required (my code ran on a device other than my dev box). I’d estimate that setting up each debug run would have taken me 5 minutes with a real h/w and it was more like seconds with unit tests. Over the course of three months, this saved hundreds of hours of my time.

When my first release of the TiVo code wrapped up, I was in-between assignments, so I wrote a “season subscription” feature. This feature would periodically scan the device for new shows and download them if they were part of a season that you wanted. By this time, the test team and marketing had taken my TV and all my TiVOs. I ended up spending 3 days writing all the code and only validating with new unit tests. I then just emailed the new installer to the test group and everything worked fine on the hardware.

Catch Issues Before They Get Shipped

This is the one part of unit testing that you will likely hear from a manager armed with a “quality” slide. It is in fact true. Bugs are cheaper to fix earlier, customers are happier, and better yet, bugs or quirks in your API don’t become an implicit part of the API.

Allow Me to Really Understand an API/Project

If you are new to a project and really want to understand how the API works, there’s no better way than writing unit tests. You can really learn the API, the “quirks” of the code, and help the project out at the same time.

Broken Unit Tests

Unit tests are only useful if test failures are taken seriously, and they’re only taken seriously if the build (hopefully continuous) breaks when they fail. Never underestimate the ability of people to forget (intentionally or otherwise) to run unit tests or to ignore failures. The tests I worked on for the Ubuntu Accomplishments Daemon had been broken for months and were so outdated they had to be scrapped completely. The new tests will run during every build which is run at least daily. If the tests fail, the deb is not built.

Ubuntu Accomplishments Daemon Unit Tests

I’ve spend the last week or so writing unit tests for the Ubuntu Accomplishments Daemon. You can see the pile of tests here in the tests.py file. While writing the tests we found about 5 bugs in the accomplishments-daemon, most were small fixes, but most of them also caused the API to abort, which would kill the daemon in a real environment. Making the daemon more resilient will make Ubuntu Accomplishments much friendlier to end-users.

There are still plenty of APIs that are not yet tested, so if you’d like to dive in, please pull the code and look at the to do list at the top of tests.py.

Conclusion

There are entire books written about unit testing, about how to do it, and about why. However, if you’re like me, you’ll need to see the benefits first-hand to really want to write unit-tests. So go read the books, and then give it a try, I promise it will reduce your bug count and improve your speed of execution.

Tagged , , ,

How Does udev Know What’s a Keyboard or Mouse?

Last week, I found myself having to dive into the udev code to figure out how it determines whether what is a mouse or a keyboard. To solve the problem I was working on, I ended up having to replicate some of that logic in python, which is posted at the bottom. Let me explain how it works.

Device Info in /sys

To start, let’s look in /sys and see what input devices we have. If you look in /sys/class/input on your system, you’ll see many symlinks to devices. I’ll pick /sys/class/input/event12 to look at in more detail, so cd to event12/device

mfisch@caprica:/sys/class/input/event12/device$ ls -al
total 0
drwxr-xr-x 7 root root 0 Jul 8 16:05 .
drwxr-xr-x 3 root root 0 Jul 8 16:05 ..
drwxr-xr-x 2 root root 0 Jul 8 16:05 capabilities
lrwxrwxrwx 1 root root 0 Jul 8 20:50 device -> ../../../serio1
drwxr-xr-x 3 root root 0 Jul 8 16:05 event12
drwxr-xr-x 2 root root 0 Jul 8 16:05 id
-r--r--r-- 1 root root 4096 Jul 8 20:50 modalias
drwxr-xr-x 3 root root 0 Jul 8 16:05 mouse0
-r--r--r-- 1 root root 4096 Jul 8 16:05 name
-r--r--r-- 1 root root 4096 Jul 8 20:50 phys
drwxr-xr-x 2 root root 0 Jul 8 20:50 power
-r--r--r-- 1 root root 4096 Jul 8 20:50 properties
lrwxrwxrwx 1 root root 0 Jul 8 16:05 subsystem -> ../../../../../../class/input
-rw-r--r-- 1 root root 4096 Jul 8 16:05 uevent
-r--r--r-- 1 root root 4096 Jul 8 20:50 uniq

There are some interesting things in here, but the two I’m interested in are “name” and the “capabilities” directory. Let’s look at name first.

mfisch@caprica:/sys/class/input/event12/device$ cat name
PS/2 Generic Mouse

Using Capabilities

Okay, so this looks like a mouse, but udev doesn’t use the name to figure this out, it uses the capabilities directory. Let’s look there:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ ls -al
total 0
drwxr-xr-x 2 root root 0 Jul 8 16:05 .
drwxr-xr-x 6 root root 0 Jul 8 16:05 ..
-r--r--r-- 1 root root 4096 Jul 8 16:05 abs
-r--r--r-- 1 root root 4096 Jul 8 16:05 ev
-r--r--r-- 1 root root 4096 Jul 8 20:43 ff
-r--r--r-- 1 root root 4096 Jul 8 16:05 key
-r--r--r-- 1 root root 4096 Jul 8 20:39 led
-r--r--r-- 1 root root 4096 Jul 8 20:43 msc
-r--r--r-- 1 root root 4096 Jul 8 16:05 rel
-r--r--r-- 1 root root 4096 Jul 8 20:43 snd
-r--r--r-- 1 root root 4096 Jul 8 16:05 sw

The aptly named capabilities provide information to udev on what “capabilities” the device has. udev is specifically interested in the following ones: abs (Absolute axes), ev (Event types), key (Keys and Buttons), and rel (Relative axes). Let’s examine one of these to determine what data it contains:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ cat ev
7

So what does 7 mean? This is a bitmask (111) who’s bits are defined in /usr/include/linux/input.h. That file has bits defined for each of the types listed in the capabilities directory, the #define prefix matches the name in the capabilities directory, so we’re looking at EV_ (Note: key is represented by defines KEY_ and BTN_, since it’s for keys and buttons).

Since this mask is 0x7 or 111, we have bits 0, 1, and 2 set and the rest are false. Judging from the #defines, the following bits are set for this device:

#define EV_SYN 0x00
#define EV_KEY 0x01
#define EV_REL 0x02

This tells us that our device gets key events, relative movement event, and synchronization events. (Figuring this out required poking around the header a bit more).

A Quick Note About the ‘key’ Capability

The entry ‘key’ is much longer than ev and requires more explanation. Let’s look at what the ‘key’ capabilities shows for my keyboard:

20000 20000000020 0 0 500f02100002 3803078f900d401 feffffdfffefffff fffffffffffffffe

So that’s much larger than ev was. It’s still a bitmask, but it’s been split into words, using the word-size sense, so 8 bytes on my system. Since this mask is little endian, the ‘20000’ represents the highest set bits. This makes processing need to take place in word-size chunks, starting at the right. Another way to think of the output above is to think about it as one large mask, as long as you pad each “word” to be 16 digits, for example, what you really see above is this:

00000000000200000000020000000020000000000000000000000000000000000000500f02100002 03803078f900d401feffffdfffeffffffffffffffffffffe

To solve my problem, I split the output up on the spaces and set 64-bits at a time in my mask starting from the right-most word.

Now that we’ve read all these bitmasks, we need to do something with it.

What udev Does with this Info

If you look at the udev source code, specifically udev/udev-builtin-input_id.c, you can see how this data is used.

For mice, the code is fairly straightforward, look at the code for the test_pointers function. We really want to know where they set “is_mouse”. The code is also looking for other stuff like touchpads, so let’s ignore that. It basically boils down to this:

if EV_REL and REL_X and REL_Y and BTN_MOUSE:
you have a mouse

There’s also a similar block where you can replace REL above with ABS, which the comment claims is for VMWare’s USB mouse, but I kept it in my solution because I didn’t quite trust the comment and it seemed harmless otherwise.

For keyboards, the code is a bit more complex because it’s basically checking to see if you have a bunch of keys defined, this code is in the function test_key. then test_key looks for a keyboard it boils down to this:

If any bits are set in 'ev' and
if bits 1-31 (but not 0) are set:
you have a keyboard

Note: bits 1-31 represent the Escape key, numbers, and Q through D.

Putting it All Together

So now that we know how it works, let’s see it in action! Here it is run on my dev box:

mfisch@caprica:~/tmp/find_input$ ./find_input.py
INFO:root:/dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root:/dev/input/event12 is a mouse (PS/2 Generic Mouse)

Here it is run on another system with more devices attached:

INFO:root: /dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root: /dev/input/event13 is a mouse (PS/2 Generic Mouse)
INFO:root: /dev/input/event15 is a keyboard (BTC USB Keyboard)
INFO:root: /dev/input/event16 is a mouse (Primax HP Wireless Laser Mini Mouse)

My python code including unit tests! is avaialble in bzr at lp:~mfisch/+junk/find_input

Comments, bugs, and fixes all gladly accepted.

Tagged , , ,

Forcing lightdm to Use a Specific Resolution

This morning I was debugging a problem in unity-greeter and I suspected that it might be related to the resolution of the system I was using. Since I was live booting it was a pain to make changes, mainly because I didn’t have all my standard tools (or web access). I ended up messing around with xorg.conf files for awhile, without much success (this reminded me of college, but not pleasantly). After some other searches, I ended up settling on this method:

For this example, I wanted a resolution of 1024×600.

  1. Login to my system
  2. Open a terminal and run cvt 1024 600
  3. Copy the resulting output, minus ModeLine portion
  4. mfisch@caprica:~$ cvt 1024 600
    # 1024x600 59.85 Hz (CVT) hsync: 37.35 kHz; pclk: 49.00 MHz
    Modeline "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync
    
  5. Run xrandr –newmode
  6. mfisch@caprica:~$ xrandr --newmode "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync
    
  7. Run xrandr –addmode LVDS 1024x600_60.00. (Note that there are several displays that xrandr shows, I wanted to change my main laptop display, LVDS, you may want to change another. Run xrandr without arguments to see all the displays.)
  8. Run xrandr –output LVDS –mode 1024x600_60.00
  9. If that switches your resolution and looks right, then lets make it permanent.
  10. Edit: /etc/lightdm/lightdm.conf, add this line (note you can choose a different path:
  11. display-setup-script=/usr/bin/lightdmxrandr
  12. Make a new file called /usr/bin/lightdmxrandr, make it a shell script that just executes lines 4, 5, and 6 from above
  13. #!/bin/sh
    xrandr --newmode "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync
    xrandr --addmode LVDS 1024x600_60.00
    xrandr --output LVDS --mode 1024x600_60.00
    
  14. Make /usr/bin/lightdmxrandr executable, logout, and restart lightdm (sudo initctl restart lighthdm)
  15. Enjoy your new resolution!

Here are the two links that helped me piece this together:

How change display resolution settings using xrandr
How can I make xrandr customization permanent?

Tagged , ,

Sending Custom Data with DHCP Option Tags … and Then Doing Something with the Data

Over the past week I’ve been playing with DHCP option tags, and using them to send data to DHCP client systems, and then do something with the data. I didn’t know much about DHCP option tags before doing this, I figure DHCP just gave you an IP address and maybe a nameserver. In reality, DHCP can be customized to send a couple hundred different settings, including where the IRC or SMTP servers are, VOIP settings, and lots more. What makes it really powerful is that you can use the unassigned options to create your own settings, and then do actions on the client when they get set. In this simple example, I’m going to send and set a new “message of the day” via dhcp. The message of the day is a (usually) short text message shown when you login via the terminal and is stored in /etc/motd. If you are trying this on a production system, you should back-up your /etc/motd first.

Server Setup

On the server, after installing dhcpd, edit /etc/dhcp/dhcpd.conf, create a valid subnet and then in the global space, add your new option. Since options 151-174 are unassigned, I will use option 151. You could also use a ‘site-local’ option between 224-254.

Add the following outside any subnet sections:
option motd code 151 = text;
option motd "Happy Easter from your DHCP server team";

Client Setup

On the client, manually run dhclient -d and make sure you’re getting a valid IP from the server. Once you know that is working, edit /etc/dhcp/dhclient.conf and add two parts, first you need to name the new option and second you need to request it:

Add this to the top, you may see another option code there, add it near that:
option motd code 151 = text;

Then look for the big request block, add your new option into that block:
request subnet-mask, broadcast-address, time-offset, routers,
domain-name, domain-name-servers, domain-search, host-name,
netbios-name-servers, netbios-scope, interface-mtu,
rfc3442-classless-static-routes, ntp-servers,
dhcp6.domain-search, dhcp6.fqdn,
dhcp6.name-servers, dhcp6.sntp-servers,
motd;

Now, let’s see if we get it. Edit /etc/dhcp/dhclient-exit-hooks.d/debug and switch RUN=”no” to RUN=”yes”. Then edit the large block of variables and add your new option, motd.

Re-run dhclient -d and then check /tmp/dhclient-script.debug, do you see your motd option? If so, your setup is correct. It should look liks this:

new_motd='Happy Easter from your DHCP server team'

Doing Something When You Get an Option

dhclient defines exit-hooks, one of which is the debug script you edited above. You can create your own shell script in /etc/dhcp/dhclient-exit-hooks.d/ and follow some of the other examples in there. DHCP options come in as variables, so in this case, the script would use ${new_motd}. From what I can tell everything in here has to be a shell script, a python script I tried did not work. Here is my script to set motd:

if [ -n "${new_motd+x} ]; then
echo ${new_motd} > /etc/motd
fi

Re-run dhclient -d and you should see that the script has run and your message of the day is now a Happy Easter message.

What About NetworkManager?

If you are running NetworkManager on your system, and you probably are, then NetworkManager starts dhclient during boot and subverts the running of dhclient’s exit hook scripts. You can use NetworkManager’s own version of “exit hooks” which live in /etc/NetworkManager/dispatcher.d to respond to DHCP events. Scripts in that directory can be written directly in python. The scripts run from NetworkManager’s dispatcher use slightly different environment variables for DHCP options, they’re in all caps and are pre-pended with DHCP4_ (and presumably DHCP6_ if you’re serving IPv6 addresses).

if [ -n "${DHCP4_MOTD+x} ]; then
echo ${DHCP4_MOTD} > /etc/motd
fi

See the man NetworkManager(8) for more details on actions that your scripts can respond to.

Options as Expressions

There are more advanced things you can do. For example, you set a setting on the server based on an expression, for example:

if option dhcp-user-class = "fortcollins-office" {
option smtp-server "smtp.ftc-office.net";
}

You then configure the client to send up a dhcp-user-class of “fortcollins-office” for all your PCs in Fort Collins.

Using my example, you could change the holiday in the message based on the date that the lease was given out.

See man dhcp-eval(5) for more info on expressions.

Encapsulated Options

Another thing you can do is vendor encapsulated options. Vendor encapsulated options take a bunch of vendor specific info and encapsulate it into one option. This means that you can pack a bunch of data into one option. I am not sure if all dhcpd servers support this construct however.

Testing This
If you want to try DHCP options without messing up your network, the quickest way is to create two virtual machines, a client and a server, and give them each two NICs. One NIC on each machine should be on a private network, and the server should have a static IP on this private network NIC and serve DHCP on that interface. The other non-private NIC is used by you to copy files to and from your virtual machines, but is otherwise not specifically required. I did this setup using Ubuntu 12.04 precise and qemu and it works great.

Conclusion
Customized DHCP options let you transmit data to your DHCP clients and then write scripts to act on that data. Although my example is contrived, you could use DHCP to transmit useful system settings and then configure them via script.

References

Tagged , , ,

Easily Create a Default Settings Package with Ubuntu Defaults Builder

If you’re rolling your own Ubuntu edition, usually a localized one, then you typically need to configure a bunch of settings. Previously this was done by creating a config package and overwriting settings files manually. This was a pain and also required knowledge of which settings to modify and where and how they were stored. Ubuntu-defaults-builder solves most of these issues for anyone rolling their own Ubuntu by abstracting the way settings are saved for you, you edit a simple text file and let the tools generate the settings you need.

Here’s How it Works

After installing ubuntu-defaults-builder, you create your config area by running ubuntu-defaults-template [package-name]. Package-name should be something like ubuntu-defaults-germany or ubuntu-defaults-mydistro. This creates a directory named [package-name]. Inside that directory are simple config files that you can modify:

drwxr-xr-x 12 1000 1000 4096 Mar 23 21:08 .
drwxrwxr-x 4 1000 1000 4096 Mar 23 21:05 ..
drwxr-xr-x 2 1000 1000 4096 Mar 25 04:11 debian
-rw-r--r-- 1 1000 1000 799 Mar 22 21:38 depends.txt
drwxr-xr-x 2 1000 1000 4096 Mar 23 02:34 desktop
drwxr-xr-x 2 1000 1000 4096 Mar 23 02:44 hooks
drwxr-xr-x 2 1000 1000 4096 Mar 22 21:38 i18n
drwxr-xr-x 4 1000 1000 4096 Mar 22 21:41 icons
drwxr-xr-x 2 1000 1000 4096 Mar 22 21:41 multimedia
-rw-r--r-- 1 1000 1000 871 Mar 22 21:38 recommends.txt
drwxr-xr-x 2 1000 1000 4096 Mar 23 20:53 unity
drwxr-xr-x 2 1000 1000 4096 Mar 22 21:40 webbrowser

The files are not “real” conf files, but are used by ubuntu-defaults-builder to create gconf and gsettings overrides. Most of the conf files are simple and self-documenting. Look at /unity/launchers.txt for example, in that file you specify what default launchers you want in Unity.

root@caprica:~/Projects/# cat ubuntu-defaults-fred/unity/launchers.txt
@clear
nautilus-home
firefox

Now, just modify all these files how you would like them. Finally you can build your defaults package with dpkg-buildpackage. After you do that, take a look at the .deb with dpkg -c and you’ll see that your settings have become overrides for anything that uses gconf or dconf/gsettings:

cat /usr/share/glib-2.0/schemas/ubuntu-defaults-fred.gschema.override
[com.canonical.Unity.Launcher]
favorites=['nautilus-home.desktop', 'firefox.desktop']

What Can You Configure?

ubuntu-defaults-builder, as of 0.27, lets you modify the following settings:

  • depends.txt – a list of packages to make dependencies of the defaults package, this is helpful when building the image.
  • recommends.txt – a list of packages to make recommends of the defaults package, this is helpful when building the image.
  • desktop/ directory – set the background, default MIME handlers, default user session, and adding custom desktop files.
  • hooks/chroot – extra hooks to be run during image build (advanced)
  • i18n/ directory – set default keyboard layout, language, and which language packs to include in the image
  • icons/ directory – where to place custom icons, for general use or for any custom .desktop files
  • multimedia/radiostations.txt – customize the radio stations list for rhythmbox and banshee
  • unity/launchers.txt – set the default list of launchers
  • webbrowser/ directory – append bookmarks to the bookmarks menu or toolbar, set the default search engine or startpage. Currently only Firefox is supported.

Testing

At this point you probably want to test it in a VM. However, due to the way that dconf settings work, the simplest method to ensure your changes will be seen is to test like this:

1) Copy the defaults package to the VM.
2) Install the defaults package
3) Create a new user
4) Logout
5) Login as the new user. This will ensure that all the default settings take precedence.

You could also use your VM’s snapshot model and save yourself from ending up with 50 user accounts if you do a lot of testing.

Once your defaults package is good, you can build an image with it using the command ubuntu-defaults-image. This tool allows you to specify a locale, add packages or sources, etc and will generate an install image for you to do final testing on.

Limitations

Ubuntu-defaults-builder is a great tool, but it has limitations. First and foremost is that you cannot change everything you might want to using it. Unless you’re going to write a patch for ubuntu-defaults-builder, you are limited to what the tool knows about. The tool has specialized routines for almost every setting it knows how to to. If you want to set other settings, for example, some advanced lightdm.conf settings (such as disabling guest or hiding the user list), you will have to override it using the old way of dropping in a new lightdm.conf file (or using /usr/lib/lightdm/lightdm-set-defaults if that will do what you want). For settings stored in gsettings/dconf you will have to write your own gsettings override files and use the override_dh_installgsettings rule in debian/rules.

When you are making your own overrides, be careful that you’re not interacting with the overrides that are built by ubuntu-defaults-builder. For example, in my case, I get two overrides, the one starting with 50 is from the dh_installgsettings step during the build:

-rw-r–r– 1 root root 68 Mar 23 20:18 50_ubuntu-defaults-fred.gschema.override
-rw-r–r– 1 root root 146 Mar 23 21:06 ubuntu-defaults-fred.gschema.override

I do not know the order the two files will be handled in, so I just try not to let them overlap settings. The easiest way to do this is to use ubuntu-defaults-builder’s own mechanisms whenever possible and then to double-check your output .deb file.

More Info

The spec for ubuntu-defaults-builder, which has additional details on what it does, how, and why, is available here.

Tagged , , ,