luis | # 1 | 2011-02-28, 15:18 | Report

Hi folks,

I know, not really the place to post this, but you've been quite helpful before, so.... sorry.

I want to do a task in python in a way that does not block the program. For example, downloading files from the internet.

Of course I could do a separate program and call a subprocess, or do somethink like "os.popen('wget .... &')". But I want to do this inside the main program, in pure python.

Which is the best way to do this? Can you point me to some reading?

Thanks!!

Cheers,
L.

juise- | # 2 | 2011-02-28, 18:03 | Report

You use threads for that.

Here's something to get started, and to find more keywords to google for:
http://www.devshed.com/c/a/Python/Ba...g-in-Python/1/

But beware, multithreaded programming can get complicated. So if you just need to get something done, subprocess and wget might not be such a bad idea after all...

epage | # 3 | 2011-02-28, 18:13 | Report

I generally recommend simplifying thread/lock management.

If you are doing a simple shell script, use the Queue class to retrieve results from the threads.

If you are using gobject or QtCore you deal with callbacks a lot. The problem with that is it spreads the logic of your code out. I've put together some constructs that simplify pushing things off into threads and then getting the results back. If so I can give you some pointers on them.

luis | # 4 | 2011-02-28, 19:24 | Report

@ epage and juise:

Sorry, I should have been more clear. I don't want anything too complicated. I just want my app not to get blocked while it downloads some specific files from the internet, since those files are not used by the program itself.

Will take a look on threads.

Thanks!

L.

epage | # 5 | 2011-02-28, 20:10 | Report

You say app, which implies GUI (rather than script). This adds complication because you have to notify the UI thread that data is available. I have code to simplify this for both Qt and GTK. Interested?

luis | # 6 | 2011-02-28, 21:28 | Report

Yes, my app has a gtk GUI. I only want a dialog to open when the download is finished, but that the download does not block the program.
If you have a code for something like this, yes, I'm interested.

Thanks!
L.

jstokes | # 7 | 2011-02-28, 21:33 | Report

http://www.pygtk.org/pygtk2reference...t--spawn-async

v13 | # 8 | 2011-02-28, 21:34 | Report

It depends on your program:

If you have a cpu-intensive program (let's say a password cracker - that runs all the time) that needs to communicate with the network you may need separate processes (but for the cpu-intensive part).

For most other cases you do this using select/poll or an API that is based on that (like asyncore or twisted).

On the other hand, if you're writing a GUI program you'll have to look at the toolkit's libraries for something that properly interacts with its main loop. Qt for example has a networking module that will fit your needs.

epage | # 9 | 2011-02-28, 22:03 | Report

Take a look at the "login" and "_login" functions in the following file
https://github.com/epage/The-One-Rin...ice/session.py

"login" creates a AsyncLinearExecution which runs "_login" in the foreground thread but everything at a "yield" in a background thread. Looking at the "_login" function this takes what would be two or three callbacks with different kinds of state and simplifies it to a single function.

EDIT: And since I wrote that code, I've generalized it for both Qt and GTK. Makes callback code a lot simpler. Each piece of code after a "yield" is runs as a callback and you would have had to find a way to pass all that state to that callback.

luis | # 10 | 2011-03-01, 02:16 | Report

Probably I didn't understand this, but isn't this something to call an external program? Kind of subprocess.Popen ? I want to run a method of the program itself async.

Thanks,
L.