maemo.org - Talk
Page 3 of 11 12 3 45 Last »
Show 40 post(s) from this thread on one page

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Maemo 5 / Fremantle (https://talk.maemo.org/forumdisplay.php?f=40)
-   -   [Announce] Enhanced BusyBox package (https://talk.maemo.org/showthread.php?t=72801)

iDont 2011-07-05 10:15
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by qole (Post 1044763)
iDont, I just want to say "THANKS" for this great package. The outdated, broken, and crippled shell in Maemo has been constantly driving me NUTS. I suggested it for the CSSU (since it is such a central part of the system), but I'll accept it as a standalone package as well.

I will be installing and testing this ASAP!
Whoa, thanks for the big thanks and the testing! :)

Quote:
Originally Posted by Mentalist Traceur (Post 1044782)
I finally voted on the version in Extras-testing. Sorry for how long it took me to notice that a vote is necessary. I definitely think this belongs in Extras.
Don't worry about it, I'm not in a hurry ;). Thanks for the vote up!

It's a shame though that each version of any package has to be voted up separately (i.e. I would love to get 1.18.5power2 in Maemo's extras; it supports installation in the SDK and has a less scary warning during first-time installation). Ah well, let's get 1.18.5power1 in extras first, as it already had 6 thumbs up currently.

Quote:
Originally Posted by demolition (Post 1044786)
Assuming this isn't something very specific, it would be useful to be aware of installation pit-falls and how to resolve them oneself... i.e. unless the error(s) were caused by something completely out of the blue: once this is fixed, any chance you'd post (please) a brief note on what went wrong, why and how to fix/avoid?
I see your point, and I completely agree with you. I've had one other report of an (un)install problem with busybox-power at the time of writing this, which I handled the very same way as how you like to see it (1. The report 2. My reply ("I'll contact you via PM") 3. Brief summary).

About the new problem. I actually do think this is something very specific. If I can trust these stats, then only two installations I'm aware of have failed out of >10000. One of those has been confirmed to be not caused by busybox-power failing, and the other is not confirmed yet. I haven't received a reply from awett yet, so I can't comment any further on the problem he encountered.

Anyway, for what I know, there are currently no pit-falls in the (un)installation scripts. Be assured, a lot of care has been put into them to prevent broken installations ;)

Quote:
Originally Posted by demolition (Post 1044786)
And, just to confirm - you'd like all testing comments on the package page?
Yes, I think that is the proper place for discussion regarding testing.

--
On other news: I just received an approval for a garage page for busybox-power, so a proper bugtracker and more are on their way :).
Also, I want to check if there is any interest in getting busybox-power working on the N8xx/N770. If you are interested in this, leave a message here (I'll move this message to the first/second post later, so it will not go unnoticed)

qole 2011-07-07 08:15
Re: [Announce] Enhanced BusyBox package
 
Hi iDont, is it possible to fix Bug 5317? I thought for sure that your Busybox 1.18 would fix it, but it still doesn't save my history if I press on the X instead of typing "exit". I even rebooted just to be sure!

EDIT: maybe fix Bug 7014 while you're in there?

EDIT AGAIN: Even just applying this workaround would be nice...
Quote:
It really should install a signal handler that exits the shell cleanly, but a simpler
workaround without patching the source would be something like:
Code:
$ echo "trap exit SIGHUP SIGINT SIGTERM" >> /home/user/.profile
But since you're patching the source anyway...

iDont 2011-07-07 13:01
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by qole (Post 1046262)
Hi iDont, is it possible to fix Bug 5317? I thought for sure that your Busybox 1.18 would fix it, but it still doesn't save my history if I press on the X instead of typing "exit". I even rebooted just to be sure!
This is one is a little bit complicated. Here's some background information on it:
Upstream BusyBox handles shell history perfectly well. Even Maemo's stock BusyBox handled shell history well, until bug 4174 appeared, that is. As you can read there, it changes BusyBox' shell history saving behaviour from writing out the history directly after executing a command, to saving the history upon shell exit. This, however, introduces new problems (e.g. bug 5317 you've mentioned).

It's a tradeoff we have to make here. Either we keep the proper upstream shell history handling and have a (little?) bit more wear on the eMMC and a resulting (small?) increase of power usage, or we go Nokia's way of saving shell history only upon shell exit, but then we'll have problems with concurrent shells etcetera.

Apparantly, Nokia has left out their shell history patch in Harmatten, thus the bug you've mentioned is 'fixed' for Harmatten. Bug 4174, however, is reintroduced there AFAIK.

I'd like to stick with what Nokia has decided to do for Maemo 5 in the first place, which is including the shell history patch. Some averaged numbers (/guesses) on how much wear upstream shell history handling actually creates on our eMMC's would be great, but I guess those are pretty hard to quantify.

However, to give you the freedom to choose yourself, I've compiled the latest busybox-power without shell-hist.patch; click here to find it. That build should suit your shell history needs ;). I'll keep this alternative busybox-power version up-to-date, but I currently don't have any plans to push it to the repositories.
Update: Scrap the previous line. Since busybox-power 1.18.5power3, shell-hist.patch has been dropped in favor of a new patch. This new patch fixes all mentioned problems, so there's no need to maintain the special busybox-power build. I can, however, create a new version of it without the new patch if there's any demand for it (drop me a PM).

Quote:
Originally Posted by qole (Post 1046262)
EDIT AGAIN: Even just applying this workaround would be nice...
[..] But since you're patching the source anyway...
Patching the code won't do, unfortunately. It's either saving the history at exit, or saving it after each line; both with their own pros and cons. That, or we need some code to keep the history from multiple terminal instances in memory somehow, and write it out when the last terminal instance closes. However, that's a bit out of my league I'm afraid :(

Anyhow, thanks for the workaround, I seem to have missed it in the bugreport itself earlier. I'll certainly include it in the next busybox-power release.

Quote:
Originally Posted by qole (Post 1046262)
EDIT: maybe fix Bug 7014 while you're in there?
Already fixed ;):

Quote:
Originally Posted by debian/config/config.maemo
CONFIG_FEATURE_EDITING_HISTORY=100

Mentalist Traceur 2011-07-07 14:28
Re: [Announce] Enhanced BusyBox package
 
Hmmm... Personally I'm divided. On the one hand I'm used to the shell being slightly 'wrong' in behavior since my main shell experience soure is the N900. On the other hand, if Nokia chose to go with "standard" behavior for Maemo 6 (I'm not calling Harmattan 'MeeGo' in casual conversation), that must mean even the Nokia folk think it's better to keep standard shell behavior. On the other hand it's possible they changed up something else, and as a result stock shell history saving behavior is only better in combination...

Does busybox normally keep track of what *time* every command is entered? The easiest way I can imagine to get the best of both worlds if it doesn't already keep track of time-stamps is to write to history as is 'normal' upsteam, by command, but write to a file in /tmp, thus keeping the actual history for all currently running shells in RAM and not actually written. Then you can write all the lines in the /tmp shell history into the existing shell history file, while clearing the /tmp-located file, when any of the shells are closed/exit-ed.

Honestly though, I think in the long term if we don't patch the shell to work as desired without the extra flash writes, the better choice is the default upstream behavior, not current N900 behavior (although I'm sticking with what's in the repository for now, since I'm used to it enough to not personally mind it). I'm confident if we get ourselves motivated enough, we could figure out how to do the former though (either the way I suggested above, using a /tmp history file as a middle-man, or through more low-level code-voodoo, which would be more efficient, but meh), instead of having to resort to chosing which one gets the repository spot. Which, actually, is already decently nice - thank you iDont for committing to maintaining both.

qole 2011-07-07 22:55
Re: [Announce] Enhanced BusyBox package
 
I think I'd be satisfied if iDont just added the workaround to the post-install script. It looks like you only need to check to see if there's a trap exit line in /home/user/.profile and if not, drop in
Code:
trap exit SIGHUP
That will keep the save-on-exit patch and workaround the missing save-on-SIGHUP code.

lma 2011-07-08 00:32
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by iDont (Post 1046377)
Even Maemo's stock BusyBox handled shell history well, until bug 4174 appeared, that is. As you can read there, it changes BusyBox' shell history saving behaviour from writing out the history directly after executing a command, to saving the history upon shell exit. This, however, introduces new problems (e.g. bug 5317 you've mentioned).
Feel free to blame me, I should have argued harded on the upstream bugs but I kinda lost interest :-(

Quote:
It's a tradeoff we have to make here. Either we keep the proper upstream shell history handling and have a (little?) bit more wear on the eMMC and a resulting (small?) increase of power usage
That's half the story, the other half being bug 4175 (history cross-contamination between concurrently running shells).

Quote:
or we go Nokia's way of saving shell history only upon shell exit, but then we'll have problems with concurrent shells etcetera.
The other way around. Upstream busybox now avoids shells contaminating each other's history while they are running but ~/.ash_history is still a mashup of the last n commands from all sessions.

Quote:
Some averaged numbers (/guesses) on how much wear upstream shell history handling actually creates on our eMMC's would be great, but I guess those are pretty hard to quantify.
It depends on how you use it obviously. We're talking about one eraseblock cycle every time you press enter which I guess adds up to a not-insignificant amount, but probably not enough to kill your flash by itself within the device's expected lifetime.

Quote:
Originally Posted by Mentalist Traceur (Post 1046419)
On the one hand I'm used to the shell being slightly 'wrong' in behavior since my main shell experience soure is the N900. On the other hand, if Nokia chose to go with "standard" behavior for Maemo 6 (I'm not calling Harmattan 'MeeGo' in casual conversation), that must mean even the Nokia folk think it's better to keep standard shell behavior.
There's no standard sadly (POSIX typically avoids the issue), but busybox's handling of history is rather unique AFAIK. Other POSIXy shells like bash or ksh keep their history in memory and write it out at exit (they pretty much have to since they implement fc and that will be extremely broken with mixed histories).

Quote:
Originally Posted by qole (Post 1046732)
I think I'd be satisfied if iDont just added the workaround to the post-install script.
Quick'n'dirty version in the Diablo CSSU:

Code:
line="trap exit SIGHUP"
if ! grep -F -q "$line" /etc/profile; then
  echo "$line" >> /etc/profile
fi
(/etc/profile so that root's shells are also taken care of).

Mentalist Traceur 2011-07-08 01:59
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by lma (Post 1046773)
There's no standard sadly (POSIX typically avoids the issue), but busybox's handling of history is rather unique AFAIK. Other POSIXy shells like bash or ksh keep their history in memory and write it out at exit (they pretty much have to since they implement fc and that will be extremely broken with mixed histories)
Fair enough. But I still think my suggestion is realistically feasible... Find whatever place in the source the paths to the history files are kept in the "original" source (where it writes history upon every enter), and edit those paths to place "/tmp" in front of them. And then add the code from the Nokia patch that makes them write the history out at exit (but don't override the former like the patch would), and change that code to pull data from the "/tmp/[whatever]" and put that into the standard [whatever] ash history upon every shell close. That way history accumulates from all the concurrent shells in the order the commands happen across the board, and is saved on every shell close (with the above patch/workaround by you guys, even when you hit the above X.)

Busybox is coded in C, right? iDont, where do you keep the latest source for this package (my apologies if you mentioned it in this thread already. If you have just say so and I'll go look)? If I get the time I'll take a look at the source see if I can improvise a shitty patch to do the above. (I probably can't, but I can always get my own hopes up before realizing I can't do it.)

- Edit -
NVM - it's right there in the second post.

iDont 2011-07-08 18:25
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by qole (Post 1046732)
I think I'd be satisfied if iDont just added the workaround to the post-install script. It looks like you only need to check to see if there's a trap exit line in /home/user/.profile and if not, drop in
Code:
Code:
trap exit SIGHUP
That will keep the save-on-exit patch and workaround the missing save-on-SIGHUP code.
It's living in GIT already: https://garage.maemo.org/plugins/ggi...=busybox-power.

Why wouldn't you want to trap SIGINT and SIGTERM too, by the way? I'm asking this because I went with the original workaround, which did include those signals as well.

Quote:
Originally Posted by lma (Post 1046773)
It depends on how you use it obviously. We're talking about one eraseblock cycle every time you press enter which I guess adds up to a not-insignificant amount, but probably not enough to kill your flash by itself within the device's expected lifetime.
Yeah, I pretty much thought so. Still, I would love to see something more suitable for our devices than one of the two currently available history handling extremes, which brings me to:

Quote:
Originally Posted by Mentalist Traceur (Post 1046808)
Fair enough. But I still think my suggestion is realistically feasible... Find whatever place in the source the paths to the history files are kept in the "original" source (where it writes history upon every enter), and edit those paths to place "/tmp" in front of them. And then add the code from the Nokia patch that makes them write the history out at exit (but don't override the former like the patch would), and change that code to pull data from the "/tmp/[whatever]" and put that into the standard [whatever] ash history upon every shell close. That way history accumulates from all the concurrent shells in the order the commands happen across the board, and is saved on every shell close (with the above patch/workaround by you guys, even when you hit the above X.)
You're right about this being realistically feasible. Even more, I gave your idea a shot too. Attached to this post is a patch that basically does what you've stated, which is saving the history using upstream's code, but to a temporary location. Busybox will copy the history file back to the correct user's home directory upon shell exit.
It seems to be working great here, i.e. not different from upstream BusyBox. I've tried out various combinations of multiple shell instances with multiple users (root and user), all without error.

To test out the patch: place it in busybox-power's debian/patches folder, add the filename of the patch to debian/patches/series, remove shell-hist.patch from the same file (important), and finally add:
Code:
CONFIG_ASH_HIST_BUFFER=y
CONFIG_ASH_HIST_BUFFER_PATH="/tmp/.shellhistory"
to debian/config/config.maemo at line 947. Then compile the software as usual with dpkg-buildpackage.
The temporary location for .ash_history can be changed if you want. With the above config entries, the buffer history file will be written to /tmp/.shellhistory/[subdir based on user]/.ash_history

This patch should remove any problems introduced by Nokia's patch (e.g. with concurrent shells), and avoid wear on the eMMC.

People with more advanced C knowledge, please review the patch and notify me of any mistakes/enhancements. I'm sure some things could be done better than how I did them. Also, I'm always a little bit hesitant to push 'invasive' code into production environments, so positive feedback is greatly appreciated too.

Edit: this patch is obsolete. A newer version of it has been merged upstream.

lma 2011-07-08 23:05
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by iDont (Post 1047237)
This patch should remove any problems introduced by Nokia's patch (e.g. with concurrent shells)
Those problems occur without the patch. Feel free to correct me if I'm missing something, but AFAIK the only problem the patch introduces is not saving history when killed.

Quote:
please review the patch
  • In histcopy(), check the result of the input fopen and bail out before clobbering the output file.
  • I don't agree that if the user doesn't have a ~/.ash_history, or have an empty one, or /tmp is full etc they should be locked out. Return instead of exit?
  • The tmppath directory shouldn't be created with mode 0777 (predictable filenames in world-writable directories are a bit exploitable), 0700 will do.
  • One could trigger a buffer overrun by feeding the shell a long $HOME (though that doesn't give any additional privileges). Using concat_path_file for tmppath would be safer than malloc/strncat here.

Temporal 2011-07-09 00:29
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by qole (Post 1046732)
I think I'd be satisfied if iDont just added the workaround to the post-install script. It looks like you only need to check to see if there's a trap exit line in /home/user/.profile and if not, drop in
Code:
trap exit SIGHUP
That will keep the save-on-exit patch and workaround the missing save-on-SIGHUP code.
May I ask to make it optional? I mean, I edit the .ash_history and put all the things I most use like change frequency or open new windows of filebox & and ping and powertop, and I, for one, prefer that it do not remember what I have done last session unless I hit exit...

BUT I may be happy just with editing my .profile to remove the trap sighup as looks like everybody seems to prefer that it remember and I'm the only one that do not like it much. On the PC I think it's essential, but on my phone, not much.

Thanks!

iDont 2011-07-09 10:27
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by lma (Post 1047500)
Those problems occur without the patch. Feel free to correct me if I'm missing something, but AFAIK the only problem the patch introduces is not saving history when killed.
This is what I observe after uninstalling busybox-power (and an "apt-get install --reinstall busybox", to be even more sure I'm running Nokia's version):
Quote:
Shell 1: ~ $ #command 1
Shell 1: ~ $ #command 2

Shell 2: ~ $ #command 3
Shell 2: ~ $ #command 4
Shell 2: ~ $ exit

Shell 1: ~ $ #command 5
Shell 1: ~ $ exit

Shell 3 (root): Nokia-N900:~# cat /home/user/.ash_history
#command 1
#command 2
#command 5
exit
It should be noted that .ash_history contained [#command 3, #command 4, exit] upon exiting shell 2 (as seen by root shell 3). I observed the same behaviour after a recompilation of Nokia's BusyBox, but now with shell-hist.patch left out. Thus, the concurrent shell problem appears to be a problem with the BusyBox 1.10.x series. The next version I've tried (which was the latest when I started busybox-power: 1.18.4) appends each command directly after executing the command to the history file without overwriting another shell's history, causing no problems with concurrent shells.

So you're right about Nokia's patch not introducing the problem. However, it doesn't fix it either.

Regarding your comments on my patch:
Quote:
Originally Posted by lma (Post 1047500)
  • In histcopy(), check the result of the input fopen and bail out before clobbering the output file.
Good point, will change that.

Quote:
Originally Posted by lma (Post 1047500)
  • I don't agree that if the user doesn't have a ~/.ash_history, or have an empty one, or /tmp is full etc they should be locked out. Return instead of exit?
This was my intention too (see comment: "No history > No shell"), but I seem to have missed changing the exits. histcopy() is based on a regular file copy implementation, which was to be used stand-alone. I'm sure you can figure yourself what went wrong here ;). Huge thanks for pointing this out to me.

Quote:
Originally Posted by lma (Post 1047500)
  • The tmppath directory shouldn't be created with mode 0777 (predictable filenames in world-writable directories are a bit exploitable), 0700 will do.
No excuse for this, will be changed too.

Quote:
Originally Posted by lma (Post 1047500)
  • One could trigger a buffer overrun by feeding the shell a long $HOME (though that doesn't give any additional privileges). Using concat_path_file for tmppath would be safer than malloc/strncat here.
Considering it is hard to believe someone has a $HOME > ~235 characters, and the fact that it you can't really gain anything by buffer underrunning, I thought that was an appropriate solution. I will check out concat_path_file for it though, as it can only make the code better.

Thanks for the review. I'll create an v2 when I've got the time for it (probably later today).

Quote:
Originally Posted by Temporal (Post 1047536)
May I ask to make it optional? I mean, I edit the .ash_history and put all the things I most use like change frequency or open new windows of filebox & and ping and powertop, and I, for one, prefer that it do not remember what I have done last session unless I hit exit...

BUT I may be happy just with editing my .profile to remove the trap sighup as looks like everybody seems to prefer that it remember and I'm the only one that do not like it much. On the PC I think it's essential, but on my phone, not much.

Thanks!
I think it's better to leave it in, simply because .ash_history is meant to save Ash's history. Retaining incorrect behaviour doesn't seem like the right thing to do.

Just an idea: you could add something like:
Code:
trap 'echo -e "filebox\nping\npowertop\netcetera" > /home/user/.ash_history' 0
to the top of your .profile file (note: the code spans just 1 line). This will restore your .ash_history to whatever you want upon normal shell exit (i.e. without error). You won't have to mind removing the trap line after each busybox-power upgrade then, plus it might even make your life easier :)

lma 2011-07-09 11:23
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by iDont (Post 1047666)
This is what I observe after uninstalling busybox-power [...]
Aha, you are absolutely correct, I completely missed that all this time :-( I'll look into it.

Quote:
Considering it is hard to believe someone has a $HOME > ~235 characters, and the fact that it you can't really gain anything by buffer underrunning
Actually I may have been wrong about that, I had forgotten that sudo passes user's $HOME untouched to the root command (bug 5896). I don't know if it's actually exploitable, but better safe than sorry:-)

(Yeah, I know the system is wide open anyway)

lma 2011-07-09 12:03
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by lma (Post 1047680)
Aha, you are absolutely correct, I completely missed that all this time :-(
Hm, wrong again. It was pointed out before and fixed for 1.13, but then I completely forgot about it and never back-ported it to 1.6 / 1.10. I need to go have my memory looked at :-(

iDont 2011-07-09 15:22
Re: [Announce] Enhanced BusyBox package
 
Alright, here is V2 of the patch. Changes that went into this new version:
- Fixes for lma's bullets
- Histcopy() is now a static bool. Warning messages are changed too
- Fix in exitshell() - there are cases wherein tmphistory doesn't exist at this point, so check for its existence
- Cleanups in code and CONFIG entries

Again, reviews are highly appreciated. I'll provide a precompiled deb in a minute too. Please let me know if you discover any issues with it.

Update: Attached a new version (V3) of the patch. A more elaborate post will be created later; I'm currently occupied with other stuff. Precompiled deb can be found here.

Edit: this patch is obsolete. A newer version of it has been merged upstream.

---
To clear out possible confusion about the whole issue, and to summarize what this patch is all about:

There are two problems:
Problem 1: BusyBox writes out the shell history after each command, which produces wear on the eMMC.
Problem 2: The default, old BusyBox (1.10.2) had a problem with history contamination

Events:
  • Nokia has made a patch to counter problem 1 by keeping the history in-memory until shell exit.
  • Nokia's patch was based on the old BusyBox of problem 2
  • BusyBox 1.13 fixed problem 2
  • Problem 2 was reintroduced in busybox-power, which uses BusyBox 1.18.5, because we apply (a ported version of) Nokia's patch to fix problem 1.
This new patch effectively does the same as what Nokia's patch did, but targets upstream BusyBox, thus keeping problem 2 fixed, while fixing problem 1. Let's hope everything is clear now, there were some misunderstandings (including by me) in the previous, lengthy posts.

qole 2011-07-09 17:17
Re: [Announce] Enhanced BusyBox package
 
I bet that patch will be useful upstream, too...

Estel 2011-07-09 23:38
Re: [Announce] Enhanced BusyBox package
 
Great work. We can expect this to hit repos ASAP (in fact, when sending anything into them will be possible), right?

Quote:
Originally Posted by lma
In histcopy(), check the result of the input fopen and bail out before clobbering the output file.
I don't agree that if the user doesn't have a ~/.ash_history, or have an empty one, or /tmp is full etc they should be locked out. Return instead of exit?
The tmppath directory shouldn't be created with mode 0777 (predictable filenames in world-writable directories are a bit exploitable), 0700 will do.
One could trigger a buffer overrun by feeding the shell a long $HOME (though that doesn't give any additional privileges). Using concat_path_file for tmppath would be safer than malloc/strncat here.
I can't keep myself from writing that - reading this post was one of my "epic" moments "What do i know about coding/debian/linux/computers/whatever" ;) Just dropped my jaw into ground.

vi_ 2011-07-13 12:00
Re: [Announce] Enhanced BusyBox package
 
I believe I 'may' have found a bug or something. If I try to cat a file that contains a windblows newline character '^M' I lose everything preceeding that character. I.e. if I try to cat a file containing:

Code:
aaaaaaaaaa^Mbbbbbbbbb^Mccccccccccc^Mdddddddd
I will only see dddddddd as the result. If I remove the newline characters I can see the data.

It makes sed/awk/grep'ing my way to victory really hard!


edit: yay for 'dos2unix'!

iDont 2011-07-13 14:19
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by qole (Post 1047825)
I bet that patch will be useful upstream, too...
I've submitted the patch to BusyBox' mailing list. Discussion can be found here. Please don't mind the text formatting of my emails. I've just switched to Thunderbird, which does everything a little bit different.

At this moment, it seems that there is no upstream interest in the patch. Please refer to the thread over at BusyBox' mailing list for more on this.

Quote:
Originally Posted by Estel (Post 1048001)
Great work. We can expect this to hit repos ASAP (in fact, when sending anything into them will be possible), right?
The latest version fo the patch seems to do what it should do, and do just that without any adverse effects. If no issues arise, the patch will pushed to busybox-power's GIT repostory and appear in the next release. I'm currently holding the release off so others can comment on the patch.

Quote:
Originally Posted by vi_ (Post 1050095)
I believe I 'may' have found a bug or something. If I try to cat a file that contains a windblows newline character '^M' I lose everything preceeding that character.
Thanks for reporting. However, I can't seem to reproduce this bug over here:
Code:
Nokia-N900:~# echo 'aaaaaaaaaa^Mbbbbbbbbb^Mccccccccccc^Mdddddddd' > testfile && cat testfile
aaaaaaaaaa^Mbbbbbbbbb^Mccccccccccc^Mdddddddd
Some random text files from a Windows installation posed to be no problem either. Could you attach the specific text file that gave problems on your device? Thanks.

Quote:
Originally Posted by vi_ (Post 1050095)
edit: yay for 'dos2unix'!
Guess what provides dos2unix on your device :D

lma 2011-07-13 14:37
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by vi_ (Post 1050095)
If I try to cat a file that contains a windblows newline character '^M' I lose everything preceeding that character. I.e. if I try to cat a file containing:

Code:
aaaaaaaaaa^Mbbbbbbbbb^Mccccccccccc^Mdddddddd
I will only see dddddddd as the result.
I get ddddddddccc here ;-)

That's normal. ^M is carriage return (aka CR, ASCII 13, '\r'), not newline (LF, ASCII 10, '\n'). Your terminal simply interprets it correctly and moves the output cursor to the beginning of the current line.

vi_ 2011-07-13 15:08
Re: [Announce] Enhanced BusyBox package
 
Code:
curl -A "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3" http://www.metoffice.gov.uk/weather/uk/surface_pressure.html | sed -e 's#<[^>]*>##g' | grep "chilly"
(this may only work for the next 12 hours till the site is updated). The above command should show a line that contains the word chilly (it is a weather report). You can wget the page and look at it in vim to confirm that it does in fact contain the weather report text.

However the above example DOES NOT display the afore mentioned text, it only displays the characters FOLLOWING the last '^M' ion the line. I have checked this by stripping out the yucky '^M's and having it work.

Just so ya know.

lma 2011-07-13 15:26
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by iDont (Post 1050172)
I'm currently holding the release off so others can comment on the patch.
Since no one else wants to comment, here goes. Sorry if it sounds like I'm picking on you, I'm actually very happy to see someone else looking at this problem from a different perspective :-)

Code:
copy_file(tmphistory, storedhistory, 11); /* 11 = force copy & preserve permissions */
11 is actually FILEUTILS_FORCE | FILEUTILS_DEREFERENCE | FILEUTILS_PRESERVE_STATUS, one of which is lost in translation above ;-) Why not use symbolic constants here? The code would be self-documenting, and also protected against future flag changes.

Code:
if (access(CONFIG_ASH_HIST_BUFFER_PATH, R_OK == -1))
        bb_make_directory(xasprintf("%s", CONFIG_ASH_HIST_BUFFER_PATH), 01777,
If /tmp isn't there you've got bigger problems, I would just log something to stderr and proceed with no history. Besides, a) a non-root shell wouldn't be able to create it anyway, b) the return value of bb_make_directory() is ignored, and c) the mode might be inappropriate if CONFIG_ASH_HIST_BUFFER_PATH points to a different place.

Code:
tmphistory = concat_path_file(CONFIG_ASH_HIST_BUFFER_PATH,
        xasprintf(".ash_hist_%s", user));
That's a predictable filename in a world-writable dir again. To illustrate, imagine that one of your "mates" grabs your N900 when you're not looking (and before you've had a chance to run a root shell after a reboot) and types something like "ln -s /etc/passwd /tmp/.ash_hist_root". Now imagine what happens next time you start a root shell.

Code:
close(open(storedhistory, O_WRONLY | O_CREAT, 0644));
I missed this last time, why not just use creat(2)? Come to think of it, there's no need to create storedhistory at this point, you could just creat() tmphistory directly and skip the copy - one write less :-)

Code:
if (access(tmphistory, R_OK | W_OK) == -1)
        copy_file(storedhistory, tmphistory, 11); /* 11 = force copy & preserve permissions */
This will have race conditions no matter what checks you do here, so best to just use a private per-user path as in the earlier version. Also, same comment on 11 as above.

lma 2011-07-13 15:56
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by vi_ (Post 1050216)
You can wget the page and look at it in vim to confirm that it does in fact contain the weather report text.
Ouch, that page is a mess, line-ending-wise :-(. Most of it is dos/windows-style (CRLF), but the paragraphs you're interested in are old-MacOS-style (CR), and of course you're trying to get sensible returns in a system that expects LF which doesn't help :-/

You could add a "tr '\r' '\n'" to the pipeline to translate those into LFs, or "tr -d '\r'" to simply remove them.

iDont 2011-07-21 17:12
Re: [Announce] Enhanced BusyBox package
 
I've just pushed some commits to busybox-power's GIT repository, including one to include and enable a new version of the shell history patch (albeit a bit later than intended). I'm pretty confident about this one, with the patch being modified several times already. As said before, if no issues are found with the new version of the patch, it will be included in the next busybox-power release. You can find the patch here. Corrections etc. are highly appreciated.

For those who haven't read previous pages of this thread: with the mentioned patch, BusyBox' history issues with using multiple shell instances are gone.

Quote:
Originally Posted by lma (Post 1050232)
Sorry if it sounds like I'm picking on you, I'm actually very happy to see someone else looking at this problem from a different perspective :-)
Don't worry, it didn't sound so to me :). Again, thanks for the review, much appreciated.

iDont 2011-07-23 10:50
Re: [Announce] Enhanced BusyBox package
 
I've got some great news: busybox-power is now available in Maemo's extras repository! A huge thanks goes out to all the people who have voted my package up :)

In other news, busybox-power 1.18.5power3 will be released in a week or so. As it will have some nice features over 1.18.5power2, I won't push the latter into extras-testing, but push the upcoming new busybox-power version directly into there.

Update: busybox-power 1.18.5power3 has just been pushed to extras-devel. I especially want to thank lma for providing me multiple times with feedback. I'll promote this new version to -testing tomorrow. If I may quote myself:
Quote:
Originally Posted by iDont (Post 1028457)
Lastly, I've promoted busybox-power to extras-testing [..] Please subject the package to some serious testing, so it might arrive in Maemo's extras repository sometime in the future ;)
All votes are highly appreciated. Thanks in advance.

Mentalist Traceur 2011-08-02 18:33
Re: [Announce] Enhanced BusyBox package
 
Wooo @ Extras-Devel and -testing for the new BusyBox power. What with the glorious shell history tweeks and all. I was actually surprised when opening a second terminal session on my N900 earlier today, to find the commands I had JUST typed in the first one, all in my history already. So... right.... Lack of words....

Mentalist Traceur 2011-08-08 02:42
Re: [Announce] Enhanced BusyBox package
 
Question to anyone using my /sbin/preinit script modification for getting an optional command-line at boot: do you have any issues with it ever since this last version of busybox-power came out?

Mine randomly started segfaulting on me, right at the moment the "sh" line is supposed to run, and I only noticed it a few days ago, but the only thing that changed since I last saw it working was busybox-power and my fiddling around with the R&D mode CAL area (I'm writing a small command-line program in C to change the R&D mode flags from on-device, based on qwerty12's gui program for the same purpose - but I've tried resetting the R&D mode flags using qwerty's program [which I know doesn't mess up the R&D flags, since I used his tool all this time], and rebooting, and I still see a segmentation fault in the console right at the moment the "sh" command runs in my /sbin/preinit modification.)

So, anyone getting the same behavior, or is it just me?

iDont 2011-08-10 15:58
Re: [Announce] Enhanced BusyBox package
 
Mentalist Traceur: I'm experiencing those segfaults as well. It most likely does that because either root's directory is still RO or /tmp isn't available in that stage of the boot process yet.
I wanted to reply to your PM earlier, but I hadn't had access to the internet prior to today. I'm currently outide the country, so I can't dig into the issue yet. I'll be back home in a little over a week or so.

Mentalist Traceur 2011-08-17 17:00
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by iDont (Post 1067352)
Mentalist Traceur: I'm experiencing those segfaults as well. It most likely does that because either root's directory is still RO or /tmp isn't available in that stage of the boot process yet.
I wanted to reply to your PM earlier, but I hadn't had access to the internet prior to today. I'm currently outide the country, so I can't dig into the issue yet. I'll be back home in a little over a week or so.
Sorry I just noticed this reply today. Being in America I am spoiled to expect free internet in hotels, but I know that's not the norm in a lot of places, so I appreciate you going out of your way to reply while on your trip.

The /tmp directory is mounted just before my boot-shell code executes in /sbin/preinit if you use my exact code. I don't think root file system is read-only either, because I've edited /sbin/preinit when inside the shell launched by /sbin/preinit... (Though I think the boot process errors out right after the file save, probably something to do with it trying to execute the file's code but then having the file changed right under it.) Whether or not everything in the root file system is mounted though, or any parts haven't been 'made' yet (if there's anything that's in /tmp or anywhere else that's needed for the current busybox-power but that gets dynamically made/remade/updated later in the boot process).

So, in other words, way out of my league, but there is a "mount tmpfs [...] /tmp" line right before my code in /sbin/preinit. Either way, your latest reply to me by PM today is promising because it suggests this problem won't be relevant as soon as you implement history buffering inside busybox itself instead of the original 'use a temporary file' method.

iDont 2011-08-21 13:25
Re: [Announce] Enhanced BusyBox package
 
BusyBox 1.19 has been released a little over a week ago. It contains about 9 months of much appreciated hard work by the BusyBox developers & other contributors. It sure makes a great update :)

Busybox-power has been updated accordingly: 1.19.0power1 just got pushed to Maemo's extras-devel. I'm letting it settle there for a few days before promoting it to extras-testing because of the size of the update.

Head over to http://busybox.net/ if you want to find out more about the changes that went into BusyBox 1.19. Changes specific to busybox-power include fixing the issue discovered by Mentalist Traceur in the posts above, busybox being split in busybox and busybox_root (see bug #700), and the activation of a few new applets.
I'd like to point out one particularly interesting new feature of BusyBox 1.19: Reverse history search. Whilst already common in other shells, searching your shell history by pressing Ctrl-R is now available in BusyBox' shells too.

Enjoy the new release!

Socim 2011-08-23 04:29
Re: [Announce] Enhanced BusyBox package
 
Hello.

Does v1.19.0power1 lack ipv6 support?

iDont 2011-08-23 09:40
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by Socim (Post 1075038)
Hello.

Does v1.19.0power1 lack ipv6 support?
Thanks for noticing. Since v1.19.0power1, busybox has been split into busybox and busybox_root. Both of those binaries have their own configuration file.
I apparently forgot to enable ipv6 support in the config file of busybox_root. Expect a fixed v1.19.0power2 to hit the repositories soon ;)

Mentalist Traceur 2011-08-23 22:59
Re: [Announce] Enhanced BusyBox package
 
I had a bug upgrading from busybox 1.18.5power3 to 1.19.0power2 (missed update to 1.19.0power1). History got stuck, not saving any changes, after the updates. I deleted /root/.ash_history and /home/user/.ash_history and the problem seems to be solved, just thought I'd throw out a heads-up to that effect.

iDont 2011-08-24 14:33
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by Mentalist Traceur (Post 1075533)
I had a bug upgrading from busybox 1.18.5power3 to 1.19.0power2 (missed update to 1.19.0power1). History got stuck, not saving any changes, after the updates. I deleted /root/.ash_history and /home/user/.ash_history and the problem seems to be solved, just thought I'd throw out a heads-up to that effect.
Thanks for the heads up!
I remember experiencing something similar to this too during development, but I can't seem to reproduce it now by:
  • Removing all history files
  • Downgrade to 1.18.5power3
  • Type some commands (and verify that they're in the history file)
  • Upgrade to 1.19.0power2
  • Type some new commands
which resulted in the new commands being appended to the history file as they're supposed to be. I'd love to hear back from other people whether their history is/was stuck too. If so, I might push out 1.19.0power3 to unstuck the history by some postinst magic.

I'm also having an issue with swapoff with the busybox-power 1.19.0 series, which seems to be the result of a parsing error of the options passed to it. However, busybox-power doesn't touch (i.e. patch or otherwise modify) swapoff.c and getopt32.c of BusyBox. If possible, I'd like to receive some confirmation from another busybox-power user (one or two confirmations will suffice) whether swapoff works or not for him/her. If not, I'll dig deeper in this.
Update: it seems like something broke upstream: swapoff fails with the same message using vanilla source on my laptop too.
Second update: Fixed in busybox-power 1.19.0power3 by applying the hotfixes that'll eventually make up BusyBox 1.19.1.

strange1712 2011-08-24 16:39
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by iDont (Post 1075800)
Thanks for the heads up!
I remember experiencing something similar to this too during development, but I can't seem to reproduce it now by:
  • Removing all history files
  • Downgrade to 1.18.5power3
  • Type some commands (and verify that they're in the history file)
  • Upgrade to 1.19.0power2
  • Type some new commands
which resulted in the new commands being appended to the history file as they're supposed to be. I'd love to hear back from other people whether their history is/was stuck too. If so, I might push out 1.19.0power3 to unstuck the history by some postinst magic.

I'm also having an issue with swapoff with the busybox-power 1.19.0 series, which seems to be the result of a parsing error of the options passed to it. However, busybox-power doesn't touch (i.e. patch or otherwise modify) swapoff.c and getopt32.c of BusyBox. If possible, I'd like to receive some confirmation from another busybox-power user (one or two confirmations will suffice) whether swapoff works or not for him/her. If not, I'll dig deeper in this.
Update: it seems like something broke upstream: swapoff fails with the same message using vanilla source on my laptop too. The bugreport can be found here. I don't need to receive confirmation anymore regarding swapoff in busybox-power.
Thanks, I didn't know what was that error related to. I hope it may be solved soon...

Mentalist Traceur 2011-08-25 14:53
Re: [Announce] Enhanced BusyBox package
 
This might be relevant - I THINK I had two xterms open when updating busybox, so I suppose that could have something to do with it.

pusak gaoq 2011-08-25 18:38
Re: [Announce] Enhanced BusyBox package
 
i got a bug since i update to your latest update....my device fails to reboot properly...
if i use TC & choose "auto-reboot after saving setting" the device shutdown not reboot...when i run the reboot command on x-term it also fails to reboot properly....it just shutdown my device....
i suspected if it not this it might be related to kernel power v48 aswell...

Estel 2011-08-26 04:23
Re: [Announce] Enhanced BusyBox package
 
I got no problems with manual reboot, using bot latest busybox-power and kp48.

Ho ever, other strange problems is showing up, which may be related to busybox-power *or* kp48. Could You guys please open xterm, gain root via "root", then

leafpad /usr/bin/whateveronescript_in_that_location

... then try "save as"?

In my case, it result in spontaneous reboot all the time. Furthermore, calling upon leafpad after "sudo gainroot" - instead of "root" and trying to "save as" result in leafpad crashing with "Segmentation fault" output in xterm (this time, without rebooting N900).

As odd it may sound, I'm pretty sure that this started after busybox power "big update" OR KP48. Most likely the latter, but I'm reporting here "just in case".

Also, it's possible that people with R&D mode enabled, won't experience this problem - it may be some watchdog reaction. Mentalist, I'm looking at You ;)

Temporal 2011-08-26 06:50
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by pusak gaoq (Post 1076621)
i got a bug since i update to your latest update....my device fails to reboot properly...
if i use TC & choose "auto-reboot after saving setting" the device shutdown not reboot...when i run the reboot command on x-term it also fails to reboot properly....it just shutdown my device....
i suspected if it not this it might be related to kernel power v48 aswell...
This is a Kernel Power related problem. There are at least 2 threads about this kernel power problem.
http://talk.maemo.org/showthread.php?t=71127
http://talk.maemo.org/showthread.php?t=72002

May I ask if you have the PS3 Sixaxis installed too? May seem random, but I'm just curious.

iDont 2011-08-26 19:29
Re: [Announce] Enhanced BusyBox package
 
Quote:
Originally Posted by strange1712 (Post 1075863)
Thanks, I didn't know what was that error related to. I hope it may be solved soon...
In case you've missed the last edit of my previous post: it's fixed in the latest busybox-power ;)

Quote:
Originally Posted by Mentalist Traceur (Post 1076493)
This might be relevant - I THINK I had two xterms open when updating busybox, so I suppose that could have something to do with it.
Hmm, I tried reproducing the stuck history the same way as before, only now with multiple xterms open while upgrading (both root and non-root): same result, the history didn't got stuck. No clues yet :-/

Quote:
Originally Posted by Estel (Post 1076817)
In my case, it result in spontaneous reboot all the time. Furthermore, calling upon leafpad after "sudo gainroot" - instead of "root" and trying to "save as" result in leafpad crashing with "Segmentation fault" output in xterm (this time, without rebooting N900).

As odd it may sound, I'm pretty sure that this started after busybox power "big update" OR KP48. Most likely the latter, but I'm reporting here "just in case".
I'm experiencing what you just described with my own KP48-derived kernel too. After downgrading busybox-power to 1.18.5power3 the issue persisted, so at the moment I'm leaning towards kernel-power as being the source of the issue. This post seems to support that.
I must add that I haven't tried your scenario with a pre-KP48 kernel

Estel 2011-08-26 20:42
Re: [Announce] Enhanced BusyBox package
 
No PS3 sixaxis here, but it's now 99% sure, that it's kp48 related. Thanks for quick answers and tracing!


All times are GMT. The time now is 10:38.
Page 3 of 11 12 3 45 Last »
Show 40 post(s) from this thread on one page

vBulletin® Version 3.8.8