The A-Z guide for recompiling and extending kernel-power - maemo.org

NiQ | # 1 | 2011-03-15, 18:46 | Report

kernel-power extends the capabilities of the N900 kernel by a lot, but sometimes you'd like to have some features that don't exist even in kernel-power, which means we need to compile a custom kernel. It is generally a good idea to base your custom kernel on kernel-power rather than the stock kernel in order to retain (or gain) all the nice features that it already has while adding your own.
As I spent the last week trying to compile my own kernel based on kernel-power (and adding CD-ROM drivers) and ran into virtually any possible problem, here's my definite A-Z guide for recompiling kernel-power.
So here goes:

Step 1 - Setting up your environment
You will need a Linux machine with Scratchbox and the Maemo SDK installed. If you only have Windows machines then too bad. You will need to repartition your HD and install Linux or use virtualization utilities like VMWare to have a Linux instance running. I used Debian GNU/Linux as my platform but most people recommend using Ubuntu, which is supposedly a lot more user-friendly. Anyway, It's just according to taste.
Go to the Maemo 5 SDK installation for beginners on Nokia's website and install Scratchbox, the SDK and the Nokia binaries according to the instructions. Stop just before the "Running Maemo 5 SDK" paragraph.
If installing the SDK crashes with the "Error opening terminal" message: Exit scratchbox if you're in it and perform the solution discussed here. If you're working from an X terminal, type the following commands as root:

Code:

mkdir -p /usr/share/terminfo/x
cd /usr/share/terminfo/x
ln -s /lib/terminfo/x/xterm xterm

Then drop root privileges and enter scratchbox as usual.
Important: From now on, all commands specified are to be entered inside scratchbox, not directly on your Linux machine. Entering these commands directly on your Linux machine (especially as root) will cause problems.

Step 2 - Adjust Scratchbox for kernel compilation
Create a new target called MaemoKernel with qemu-arm CPU transparency. The second command installs the armel rootstraps to the target. The last command installs the C-library, /etc, devkits and fakeroot.

Code:

[sbox-FREMANTLE_ARMEL: ~] > sb-conf setup MaemoKernel \
-c cs2007q3-glibc2.5-arm7 \
-d qemu:perl:svn:apt-https -t /scratchbox/devkits/qemu/bin/qemu-arm-sb
[sbox-FREMANTLE_ARMEL: ~] > sb-conf select MaemoKernel
[sbox-MaemoKernel: ~] > sb-conf rs MaemoKernel \
/home/<username>/maemo-sdk-rootstrap_5.0_armel.tgz
[sbox-MaemoKernel: ~] > sb-conf in MaemoKernel -edFL

The <username> above refers to your login name in the environment. If you have used the Maemo installer, the rootstraps are under your home directory. If you have performed a manual installation, the rootstrap is under the /scratchbox/packages directory. Keep this in mind when running the sb-conf command above.
Unfortunately in many cases the MaemoKernel target will end-up broken (i.e. with a lot of missing dependencies). To make sure that we have all of the required packages, perform the following commands:

Code:

fakeroot apt-get update
fakeroot apt-get -f install

If these commands fail because apt attempts to download i386 images instead of ARM, modify them as follows:

Code:

fakeroot apt-get -o APT::Architecture=armel update
fakeroot apt-get -o APT::Architecture=armel -f install

That should fix any broken dependencies. Now install the ncurses-dev package because we'll be needing it in order to configure the kernel. Type:

Code:

fakeroot apt-get install ncurses-dev

(add the APT::Architecture=armel) option if required.
Finally, install the Nokia binaries as mentioned in step 1.
Our environment is ready.

Step 3 - Obtain kernel-power sources
Add the extra repositories to scratchbox by adding the following lines to /etc/sources.list

Code:

deb http://repository.maemo.org/extras/ fremantle free non-free
deb-src http://repository.maemo.org/extras/ fremantle free

If you have extras-testing enabled on your N900, also add:

Code:

deb http://repository.maemo.org/extras-testing/ fremantle free non-free
deb-src http://repository.maemo.org/extras-testing/ fremantle free

If you have extras-devel on your N900, also add:

Code:

deb http://repository.maemo.org/extras-devel/ fremantle free non-free
deb-src http://repository.maemo.org/extras-devel/ fremantle free

Now, get a copy of the kernel-power sources by typing the following commands:

Code:

fakeroot apt-get update
fakeroot apt-get install kernel-power-source

Wait for the download to finish. You will have a file called "kernel-power-source.tar.bz2" in /usr/src.
Create a new directory for the kernel sources and switch to it:

Code:

mkdir ~/maemo_kernel
cd ~/maemo_kernel

Now extract the sources to the current directory using the following command:

Code:

bzip2 -d < /usr/src/kernel-power-source.tar.bz2 | tar -x

If all went well you will now have a subdirectory called kernel-power-source with the sources in there.
Important: Other guides use the following command to obtain the kernel-power sources:

Code:

apt-get source kernel-power

Do NOT use this command. As of the time of writing this guide it downloads a broken version of the sources that will not successfully compile. Get the sources by installing kernel-power-source.

Step 4 - Configure the kernel
First of all, enter the kernel sources directory using:

Code:

cd kernel-power-source

We would like to use the nice menuconfig to configure our kernel. Unfortunately it has problems running in scratchbox. To make it work, edit the following file with your favorite text editor:

Code:

scripts/kconfig/lxdialog/check-lxdialog.sh

Find the following lines:

Code:

# What library to link
ldflags()
{
	for ext in so a dylib ; do

Change them to read:

Code:

# What library to link
ldflags()
{
	# XXX autodetection fails in scratchbox
	echo -lncurses
	exit
	for ext in so a dylib ; do

Then run the configuration utility with this command:

Code:

make EXTRAVERSION=<extraversion> menuconfig

Note: <extraversion> is a tag that will be appended to the kernel version, uniquely identifying it. It can be anything as long as you keep it consistent throughout the code.
<extraversion> for the stock kernel is: -omap1
<extraversion> for kernel-power it is: .10power46
I used the following: .10power46-niq

The configuration utility should now load. It should already have the default configuration of kernel-power so you can modify whatever you want. If it doesn't seem to have the kernel-power configuration, type the following command:

Code:

make EXTRAVERSION=<extraversion> rx51power_defconfig

And then launch menuconfig as described above.

Step 5 - Compiling the kernel
Edit the following file with your favorite editor:

Code:

arch/arm/plat-omap/bootreason.c

Look for the following lines:

Code:

len += sprintf(page + len, "%s\n", boot_reason);

Change them to:

Code:

// len += sprintf(page + len, "%s\n", boot_reason);
len += sprintf(page + len, "pwr_key\n");

This will allow us to use a special mode for testing whether the kernel works without actually flashing it (and possibly bricking our device) later on. We will revert this change later just before actually flashing the new kernel to our device.

Compile the kernel using the following command

Code:

make EXTRAVERSION=<extraversion> bzImage

This will create the following file, which contains the kernel image:

Code:

arch/arm/boot/zImage

If it crashes with the following error:

Code:

include/linux/autoconf.h:556:1: fatal error: /drivers/gpu/pvr/pvrconfig.h: No such file or directory

Type the following:

Code:

ln -s /home/<username>/maemo_kernel/kernel-power-source/drivers /drivers

And compile again.

Step 6 - Compiling and assembling the kernel modules:
Compile the modules with the following command:

Code:

make EXTRAVERSION=<extraversion> modules

Assemble all modules into a single directory using the following commands:

Code:

mkdir ../modules
find . -name "*.ko" -exec cp '{}' ../modules/ \;
cd ../modules

Strip the debugging information from the modules to reduce their size from ~100MB (which will not fit in the rootfs) to ~9MB with the following command:

Code:

strip --strip-debug *

Great, we have everything compiled. Now it's time to test whether it works.
You can now exit scratchbox.

Step 7 - Copying module files to the phone
Note: Before you begin this step, make sure you already have kernel-power installed. If not, install it now from the app manager or by typing the following command in a root shell on the phone:

Code:

apt-get install kernel-power-settings

Launch a root shell on your phone and go to /lib/modules. Create a subdirectory named <version><extraversion>. <version> will be the current kernel version (for Maemo it is 2.6.28 as of the time of writing and will probably not change) and <extraversion> is whatever you wrote when you compiled the kernel. For instance, since I used .10power46-niq as <extraversion>, for me the directory name will be:

Code:

2.6.28.10power46-niq

Extremely important: Make sure the directory name matches the version and extra version exactly. A mismatch of even a single letter will cause your phone not to start.

Copy all the .ko files from the modules directory of your development machine (the scratchbox directory hierarchy is located inside /scratchbox on the development machine) to the directory you've just created on your phone.
Make sure the directory and all files in it are owned by root (otherwise due to security concerns the kernel won't load them). Enter the following commands as root on the phone in the
/lib/modules directory:

Code:

chown -R root:root <version><extraversion>

Also set appropriate permissions:

Code:

chmod 644 <version><extraversion>/*

Create the module metadata files by running the following command as root on your phone:

Code:

depmod <version><extraversion>

For instance, in my case:

Code:

depmod 2.6.28.10power46-niq

Step 8 - Testing the new kernel
It is now time to test our new kernel on the phone. Note that we use a special method to boot the new kernel on the phone without actually flashing it, which means that if it doesn't work, it takes nothing more than a single reboot to revert back to the previous kernel.
Obtain the Maemo Flasher-3.5 from here and install it.
Copy the zImage file containing the kernel image to the machine on which you have the flasher installed. Now shut-down your phone (give it a little time to rest because if you touch it in the few seconds after you shut down it boots right back up). Take the USB data cable that came with your phone and connect it to your computer. Now hold the letter 'U' on your phone and while holding it, connect the cable to the phone. If everything went OK, you should be seeing a Nokia logo on your phone screen with the USB symbol in the upper-right corner. This means that the device is now in flashing mode (don't worry, we don't flash yet). If the machine from which your're running the flasher is a Windows machine you will have to wait a while for Windows to install the appropriate driver if this is the first time you connect your phone in flashing mode.
Now it's time to test your new kernel. Type the following command on your machine where the flasher is installed in the flasher's directory:

Code:

flasher-3.5 -l -b -k <path-to>/zImage

Your phone will now boot with the new kernel. If your phone boots up OK and you can use it, you can now proceed to permanently flashing it with the new kernel.
Extremely important: If your new kernel does not boot or your device immediately shuts down / restarts or you experience any other problem that you have not experienced earlier DO NOT PROCEED to the next step!!! Reboot the phone immediately to revert to your previous kernel and ask here for advice.

Step 9 - Flashing the new kernel
So now it's the time to permanently install your new kernel on the phone. First, return to scratchbox and edit the following file:

Code:

arch/arm/plat-omap/bootreason.c

Revert the changes that you made to that file in step 5. Then recompile the kernel using the following command:

Code:

make EXTRAVERSION=<extraversion> bzImage

Don't turn-off the phone yet. Start a root shell and enter the following commands:

Code:

cd /lib/modules
rm current
ln -s <version><extraversion> current

This modifies the /lib/modules/current symlink to point to the modules directory of the new kernel (kernel-power modifies the /sbin/preinit script so it is not needed but we should still do it just in case something still relies on it. You can now turn the phone off.
Copy the newly created zImage from the development machine to the flashing machine (if they are not the same one). Now put the phone into USB flashing mode as described in step 8 and then flash it uing the following command:

Code:

flasher-3.5 -f -k <path-to>/zImage

Note: Do not attempt to test the new kernel with the command you used in step 8. It will not work and the device will immediately reboot, but don't worry, if the kernel you used in step 8 worked and this is the only change you've made since then once flashed it will work.

Congratulations, you now have a new kernel.
Please write back if you found this useful.

Last edited by NiQ; 2011-03-21 at 17:35. Reason: Added changing permissions / ownership of the module files.

The Following 49 Users Say Thank You to NiQ For This Useful Post:
anapospastos, arora.rohan, bingomion, blue_led, cfh11, cheve, chrget, Chrome, clasificado, debernardis, Fabry, Flynx, futtawuh, fw190, Halftux, handaxe, hardkorek, ivgalvez, jj0, joerg_rw, JorgeFX, jstokes, jurop88, laasonen, maacruz, MaddogG, Magik, Megaltariak, Mentalist Traceur, misiak, mr id, MyNokiaN900, Netweaver, Nikem, noorallah, peterleinchen, reinob, rfranz, sinaisix, slender, Switch_, ThomasAH, Tiboric, traysh, trompkins, tuminoid, vi_, woody14619

woody14619 | # 2 | 2011-03-15, 18:59 | Report

This is awesome.. but wouldn't it be better as a Wiki page? That would allow people to update it as things change over time. (And linking that wiki page here would be great too.

)

The Following User Says Thank You to woody14619 For This Useful Post:
mr id

Tiboric | # 3 | 2011-03-15, 19:02 | Report

awesome is the exact word, Thanks NiQ

NiQ | # 4 | 2011-03-16, 14:32 | Report

Thanks to everyone who replied.
woody14619 - good idea. I'll turn it into a wiki page once I'm confident I haven't forgotten anything (just edited the post and added a thing that I forgot).

The Following 2 Users Say Thank You to NiQ For This Useful Post:
joerg_rw, sinaisix

woody14619 | # 5 | 2011-03-16, 18:15 | Report

Originally Posted by NiQ View Post

woody14619 - good idea. I'll turn it into a wiki page once I'm confident I haven't forgotten anything (just edited the post and added a thing that I forgot).

That's the advantage of the Wiki, anyone who comes across something you forgot and edit/add it to the process.

Please to put a link to it in the first post when you add it to the wiki, and thanks again for putting this very important document to writing!

hardkorek | # 6 | 2011-03-17, 07:10 | Report

Great tutorial. Thanks for posting.
Putting it in wiki is good idea.

joerg_rw | # 7 | 2011-03-19, 21:36 | Report

I'm not all convinced about the "wiki, so everybody can mess around with it" idea. NiQ's approach is rather sane. I've seen too many wiki pages going bonkers by edits of users that tried to add the useless detours they took, or simply "correcting" things they didn't understand. This is more unlikely to happen if the tutorial is really good and easy to understand, but in early stages you learn to hate when that happens. So "wiki - YES! But not immediately" :-)

Thanks NiQ! Great work.

/j