maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   OS2008 / Maemo 4 / Chinook - Diablo (https://talk.maemo.org/forumdisplay.php?f=29)
-   -   dm-loop (for qole) (https://talk.maemo.org/showthread.php?t=23891)

Matan 2008-09-26 12:37

dm-loop (for qole)
 
These two modules enable dm-loop:

http://my.arava.co.il/~matan/770/dm-mod.ko
http://my.arava.co.il/~matan/770/dm-loop.ko


To use it you need dmsetup package which is available from debfarm repository. After installing the package add a symlink from dmlosetup to dmsetup, and run something like:

dmlosetup 0 /z/debian.img

Then mount /dev/dm-0 /opt


In my (very unscientific) tests it is about 4 times faster than loop, and very close to direct access speed.

qole 2008-09-26 19:49

Re: dm-loop (for qole)
 
Matan,

you rock.

qole 2008-09-27 05:16

Re: dm-loop (for qole)
 
Quote:

Originally Posted by Matan (Post 227643)
....run something like:

dmlosetup 0 /z/debian.img

Just a slight correction, the 0 is wrong, it should be loop0 instead:

Code:

dmlosetup loop0 /media/mmc1/debian.img
Quote:

Originally Posted by Matan (Post 227643)
In my (very unscientific) tests it is about 4 times faster than loop, and very close to direct access speed.

OpenOffice loads at least 2x as fast, which is a huge improvement.

Thanks so much! I've already incorporated the Matan TurboLoop (c) into my next version of Easy Debian... :D

Stskeeps 2008-10-02 08:56

Re: dm-loop (for qole)
 
Matan: Can I probe what patch you used for the 2.6.21 kernel to get this compiled?, as it's not part of the 2.6.21 kernel by default. Reason is that I would like to compile some .o's that would follow kernel updates for Deblet.

Your work is immensely useful both for qole's Easy Debian and for my "Deblet-in-a-file" (I've just managed to compile dmsetup.static, may be interesting as well for qole), where I make a Deblet system boot entirely from a file on a vfat partition.

Matan 2008-10-02 11:15

Re: dm-loop (for qole)
 
I'm using iodine in an airport, so I won't include a link, but dm-loop project has patches for 2.6.20 which apply cleanly to 2.6.21-omap1.

qole 2008-10-02 16:22

Re: dm-loop (for qole)
 
Quote:

Originally Posted by Matan (Post 229645)
I'm using iodine in an airport...

Yikes. Now there's a man who likes his free wi-fi, no matter how slow.

Stskeeps 2008-10-04 16:17

Re: dm-loop (for qole)
 
Deblet-in-a-file now also uses dm-loop :) thanks for the great work Matan.

qole 2009-06-17 04:19

Re: dm-loop (for qole)
 
Matan: Do you think you could build these modules for the Maemo 5 kernel?

Matan 2009-06-17 08:49

Re: dm-loop (for qole)
 
I am usually not interested in software without hardware to run on, so I do not expect to install Maemo 5 SDK until (and if) a Maemo 5 tablet is available.

There are patches that should apply to modern kernel, see here:

http://osdir.com/ml/linux.kernel.dev.../msg00038.html

I believe that anyone who has the SDK installed can easily patch and compile the kernel.

qole 2009-06-17 15:50

Re: dm-loop (for qole)
 
I discovered last night that the lack of a dm-loop in Fremantle is the least of my problems. I am finding it very difficult to test my Easy Debian scripts in the SDK when there's not even a "mount" command...

qole 2009-10-22 16:47

Re: dm-loop (for qole)
 
qwerty12 is my first tester of the easy-chroot stuff in Fremantle, and he noticed the missing dm-loop.

I would like someone to upload the dm-loop kernel modules to Extras for both Diablo and Fremantle. It really isn't right that I'm packaging this as part of my easy-chroot package; my package should depend on this package.

At the end of September, Attila77 graciously offered to get the Diablo dmloop package into the repositories for me... He wrote to me:

"Just to let you know, I got the module built, but there is a bit of a snag, the i386 part of the autobuilder process dies a horrible death as it can't really deal with the arm kernel stuff and thus marks the whole package as failed. I already wrote Jeremiah about this, so hopefully with a little help I'll have this in extras shortly."

Here's a link to his package (currently in my repository, doesn't seem to be in Extras yet).

So this looks like it will be quite a challenge for anyone willing to try it for Fremantle... And can anyone help getting the package to build for Diablo?

attila77 2009-10-22 16:54

Re: dm-loop (for qole)
 
If it means something, I can 'fake' the i386 part, so only the ARM module gets built. Not pretty, but it would work with extras until Jeremiah gets some time to play with it.

BTW I though you said 'simple' loop is good enough in Fremantle ?

qwerty12 2009-10-22 16:58

Re: dm-loop (for qole)
 
Quote:

Originally Posted by attila77 (Post 355630)
If it means something, I can 'fake' the i386 part, so only the ARM module gets built. Not pretty, but it would work with extras until Jeremiah gets some time to play with it.

Well, setting the Architecture field in debian/control to armel ensures that it only gets built for ARM; no "faking" required. TBH, I don't see why the modules would be needed in an I386 Scratchbox anyway; Scratchbox insists on reporting your hosts' kernel version when running "uname -a" so I doubt they're gonna work either way.

P.S. I'd say that the modules are needed for Fremantle. "apt-get install nokia-binaries nokia-apps" completed when I woke up in the morning... :)

attila77 2009-10-22 17:13

Re: dm-loop (for qole)
 
I know about (and set) that field, but the autobuilder DOES make (or rather, tries to make) something in i386, and that's where it blows up (complains about the cross-compiler, but the actual error is not there, it shouldn't get that far in the first place)

qwerty12 2009-10-22 17:16

Re: dm-loop (for qole)
 
Quote:

Originally Posted by attila77 (Post 355644)
I know about (and set) that field, but the autobuilder DOES make (or rather, tries to make) something in i386, and that's where it blows up (complains about the cross-compiler, but the actual error is not there, it shouldn't get that far in the first place)

Seeing as you will have already once uploaded it to the autobuilder, would it be too much trouble to ask for the source package so I can look at it (not to upload with my name as the Maintainer ;)), please?

qole 2009-10-22 18:06

Re: dm-loop (for qole)
 
Thanks guys.

It's because of you and this stuff that you do that I love this place.

<warm fuzzies>

attila77 2009-10-22 19:26

Re: dm-loop (for qole)
 
Quote:

Originally Posted by qwerty12 (Post 355648)
Seeing as you will have already once uploaded it to the autobuilder, would it be too much trouble to ask for the source package so I can look at it (not to upload with my name as the Maintainer ;)), please?

Sent. I believe the thing is quite self-explanatory, I just extract the kernel source, patch the dmloop stuff, compile, and that's about it. It's probably failing because of something trivial (=me not seeing something 'cause my eyes get in the way). Under a VM on my ancient hardware, every attempt was ~30 minutes so I lost interest in the trial and error method real quick, especially as I had my armel.deb :) You are very welcome to become maintainer :)

PS The architecture is 'all' in the archive, which is wrong, I know, I was just trying to figure out a few arch related things and it remained that way in the last package.

PPS Also, Matan showed some interest in packaging modules, so if you figure out what I was missing, post here so anyone trying to add a separate deb module can work it out.

qwerty12 2009-10-23 15:45

Re: dm-loop (for qole)
 
https://garage.maemo.org/pipermail/e...er/010773.html =)

attila77 2009-10-23 15:53

Re: dm-loop (for qole)
 
qwerty12 for president ! :D

qole 2009-10-25 21:40

Re: dm-loop (for qole)
 
Hi guys,

I've finally had a chance to look at the package that you built. It has the kernel modules, but now we need the dmsetup package in Extras too (see first post in the thread).

The dmsetup package should additionally have a symbolic link made (in postinst), /sbin/dmlosetup, which points to /sbin/dmsetup.

As soon as these things appear in the repositories, I can start depending on them for my package instead of providing them.

qole 2009-10-25 21:51

Re: dm-loop (for qole)
 
What extra things would be required to add encryption here? Is yet another kernel module needed?

I could see many use-cases for an encrypted file system inside an image file on your mobile device.

attila77 2009-10-25 21:59

Re: dm-loop (for qole)
 
I'll try and push dmsetup into extras tonight barring some devmapper dependency funk.

attila77 2009-10-26 00:24

Re: dm-loop (for qole)
 
Code:

[2009-10-26 02:15:06] Processing package devmapper 1.02.27-4. Uploader: achipa, builder: builder1
[2009-10-26 02:15:11] Building devmapper 1.02.27-4 for target 'maemo-diablo-armel-extras-devel'
[2009-10-26 02:18:03] OK
[2009-10-26 02:18:06] Building devmapper 1.02.27-4 for target 'maemo-diablo-i386-extras-devel'
[2009-10-26 02:20:38] Signing build results
[2009-10-26 02:20:40] devmapper 2:1.02.27-4 has been queued for loading into diablo extras-devel repository
[2009-10-26 02:20:41] You can find more info about this build here: https://garage.maemo.org/builder/diablo/devmapper_1.02.27-4/


allnameswereout 2009-10-26 00:50

Re: dm-loop (for qole)
 
Quote:

Originally Posted by qole (Post 357873)
What extra things would be required to add encryption here? Is yet another kernel module needed?

I could see many use-cases for an encrypted file system inside an image file on your mobile device.

The dependancies I mentioned here. Instead of dmsetup I recommend cryptsetup-luks (aka cryptsetup nowadays) because it only has that one purpose. Just check out each of their manuals...

Together with an Ext2/Ext3 driver this should be compatbile with FreeOTFE, making it possible to even access r/w mode of both the image as well as the filesystem on a little endian machine such as Windows/x86-32. Like TrueCrypt. That means, that for example the image can be put on MicroSD, and a Windows/x86-32 user (most used desktop platform...) can write stuff onto it.

qole 2009-10-26 02:25

Re: dm-loop (for qole)
 
So now, in theory, you can get your own turbo loop in Diablo without needing to install easy chroot. I'll test it out and get back to you...

qole 2009-10-26 03:18

Re: dm-loop (for qole)
 
Agh! post-removal script for dmsetup is broken! Please fix!

(I expected the conflict with easy-chroot; I didn't expect the package to get "jammed" trying to remove!)

Code:

Selecting previously deselected package dmsetup.
Preparing to replace dmsetup 2:1.02.27-4 (using .../dmsetup_2%3a1.02.27-4_armel.deb) ...
Unpacking replacement dmsetup ...
dpkg: error processing /var/cache/apt/archives/dmsetup_2%3a1.02.27-4_armel.deb (--unpack):
 trying to overwrite `/sbin/dmsetup', which is also in package easy-chroot
dpkg (subprocess): unable to execute new post-removal script: Exec format error
dpkg: error while cleaning up:
 subprocess post-removal script returned error exit status 2


attila77 2009-10-26 10:43

Re: dm-loop (for qole)
 
Whoops :o Knew that softlink shouldn't be in postinst/rm... New package will be on it's way shortly.

attila77 2009-10-27 01:46

Re: dm-loop (for qole)
 
Upon further inspection, this wasn't caused by the softlink... the hacked version of dmsetup in the qole.org repo did not use the udev rules but simply ignored that part of the original package. Now, I fixed the failure and you DO get the udev rule as per the original package (can't be a bad thing, right ?). Yell if this is not what you wanted.

PS. As it's about 2:40 AM here and my eyes are closing, I first uploaded the package to the Fremantle queue by mistake. :o The fun part is that it actually went through the autobuilder without a hitch, so... we have devmapper in Fremantle :D

EDIT: If you don't know how to solve the postrm/exec format problem, just do a
Code:

rm /var/lib/dpkg/info/dmsetup.postrm
and all will be well

qole 2009-10-27 06:45

Re: dm-loop (for qole)
 
Looks like everything installs OK in Diablo. Kernel modules load, dmlosetup works, and I can mount an image just fine.

One little error when installing the new packages:
Code:

Setting up libdevmapper1.02.1 (2:1.02.27-maemo1) ...
ldconfig: /usr/lib/libgailutil.so.18 is not a symbolic link

Just for fun, I tried dmlosetup in Fremantle. It needs the kernel drivers. :(

So someone still needs to build the drivers for kernel version 2.6.28-omap1 (armv71).

qwerty12 2009-10-27 08:17

Re: dm-loop (for qole)
 
Quote:

Originally Posted by qole (Post 358800)
So someone still needs to build the drivers for kernel version 2.6.28-omap1 (armv71).

I'm certainly not going to attempt an autobuilder build when the SDK kernel source is out of date compared with what's on the device itself...

attila77 2009-10-27 22:31

Re: dm-loop (for qole)
 
Quote:

Originally Posted by qwerty12 (Post 358824)
I'm certainly not going to attempt an autobuilder build when the SDK kernel source is out of date compared with what's on the device itself...

Yeah, that one was a headscratcher - kernel on a preproduction device newer than the 'final' SDK... Or are we going to have torrent style final-ultimate-repack SDKs ? :D

qole 2009-10-27 22:36

Re: dm-loop (for qole)
 
Well, until someone attempts a build of dm-mod.ko and dm-loop.ko for Fremantle, you'll have to putter along with the "regular" loop... :(

attila77 2009-10-27 22:57

Re: dm-loop (for qole)
 
That is clear, it's just that it seems things are still in flux on the kernel level. Until the device-sdk-autobuilder trinity settles on a common 'final' version, it's not really worth the effort.

allnameswereout 2009-10-31 15:36

Re: dm-loop (for qole)
 
Hmm, what I find strange, is that dm-loop is not in mainline kernel. What is reason for this? I can hardly find information about it, and cannot find module pre-compiled either.

I have in fact 2 dm modules loaded (no idea why but they're loaded): dm_crypt and dm_road but they use _ underscore not - stripe. WTF?

qole 2009-11-16 18:07

Re: dm-loop (for qole)
 
Ok! Quim has announced the new final-final SDK, and it should have the shipping kernel version in it. Can one of you guys work your magic and get the dm-loop packages into Fremantle extras-devel? I would be so grateful!

qole 2009-11-17 18:22

Re: dm-loop (for qole)
 
Just a visual thank-you to Matan, qwerty12 and attila77...

http://farm3.static.flickr.com/2494/...5984d011_o.png

Look at those beautiful, wonderful dependencies. Sweet.


EDIT:
I've promoted the dm-loop stuff to Extras, because I need them for my packages...

qole 2009-12-02 03:37

Re: dm-loop (for qole)
 
Attila? qwerty12? Can someone fix Diablo dmsetup, please?

attila77 2009-12-02 11:29

Re: dm-loop (for qole)
 
Errr... I thought I fixed it a while back ? I did not test extensively but a quick test run went through OK... Can you confirm the problem with the latest dmsetup in extras-devel ? On a side note, now that the release firmware is out, I planned on a little Fremantle dm_loop hackery tonight.

qole 2009-12-02 17:59

Re: dm-loop (for qole)
 
Quote:

Originally Posted by attila77 (Post 404513)
Errr... I thought I fixed it a while back ? I did not test extensively but a quick test run went through OK... Can you confirm the problem with the latest dmsetup in extras-devel ?

Someone else reported the same error when trying to install the dmsetup package. Did you fix this one?

Quote:

Unpacking dmsetup (from .../dmsetup_2%3a1.02.27-maemo3_armel.deb) ...
rm: cannot remove '/etc/udev/rules.d/65_dmsetup.rules': No such file or directory
dpkg: error processing /var/cache/apt/archives/dmsetup_2%3a1.02.27-maemo3_armel.deb (--unpack):
subprocess pre-installation script returned error exit status 1
Errors were encountered while processing:
/var/cache/apt/archives/dmsetup_2%3a1.02.27-maemo3_armel.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

qole 2010-03-01 18:15

Re: dm-loop (for qole)
 
I finally tried out the Fremantle packages in Extras-devel. In fact, I've made my Fremantle Easy Chroot package use the dmloop system if it is installed.

Some things I would like:
  • I wish kernel-module-dmloop depended on dmsetup and libdevmapper1.02.1. It would be nice if the dependencies went both ways, but it is essential to pull in dmsetup if you install kernel-module-dmloop.
  • I was doing some testing, uninstalling and reinstalling the dmsetup packages in Fremantle. I had the same problem with that dmsetup.rules file not being deleted when the package is removed.


All times are GMT. The time now is 21:38.

vBulletin® Version 3.8.8