maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   MeeGo / Harmattan (https://talk.maemo.org/forumdisplay.php?f=45)
-   -   online newspaper download scripts (https://talk.maemo.org/showthread.php?t=82091)

aRTee 2012-02-03 22:40

online newspaper download scripts
 
Just to share some stuff I've been doing..

I live in Switzerland, where there are a few free newspapers popular amongst commuters; if you want a (free downloadable) newspaper from elsewhere, adapt as required.

First example: 20minutes - it becomes available at some point in the morning, I just guessed it would be there around 6am, so I setup a cron job like this:

Code:

0 6 * * mon,tue,wed,thu,fri devel-su user -c /home/user/dldepapers/getlatest20min.sh
to run the following script, which is obviously at /home/user/dldepapers/getlatest20min.sh :

Code:

#!/bin/sh

INITIEEL=uit

if [ `gconftool-2 -g /system/osso/connectivity/network_type/restricted_mode` == "false" ]
  then
    INITIEEL=al_aan
  else
    gconftool-2 -t bool -s /system/osso/connectivity/network_type/restricted_mode false
    sleep 10
fi

cd /home/user/dldepapers
DATUMVANDAAG=`date +%Y%m%d`
echo `date`: cronjob by `whoami` >> /home/user/dldepapers/download.log

if [ -e /home/user/MyDocs/Downloads/ZH_$DATUMVANDAAG.pdf ]
then
 echo "file ZH_$DATUMVANDAAG.pdf exists in the Downloads folder" >> /home/user/dldepapers/download.log
else
 echo "file ZH_$DATUMVANDAAG.pdf doesn't exist, trying to download" >> /home/user/dldepapers/download.log
 cd /home/user/MyDocs/Downloads
 wget http://www.20min.ch/printpdf/ZH_$DATUMVANDAAG.pdf
fi

if [ $INITIEEL == "uit" ]
  then
    sleep 30
    gconftool-2 -t bool -s /system/osso/connectivity/network_type/restricted_mode true

For those in Switzerland wanting a different version, just check the epaper page to see what the proper name should be, and adjust as required (ZH_ is for Zurich, LU_ is for Lucerne, etc).

As I indicated in another thread, the gconf commands serve to allow background data connections, upon which it will take just under 10 secs to get connected to my home wifi (any known wifi node, or even cellular data, if you have it set to auto connect to that). If that setting was off, it will be turned off again after half a minute.

Oh, for wget you need to do
apt-get install wget


Now, once this works, the pdf shows up in the documents list (documents button/app), but in case there was no network connection at the required moment, I created this script:

Code:

#!/bin/sh

cd /home/user/dldepapers
DELTA=0
WEEKDAG=`date +%u`
UURVDAG=`date +%H`
if [ $WEEKDAG == 6 ]
  then DELTA=86400
  #echo 1 dag
elif [ $WEEKDAG == 7 ]
  then DELTA=172800
  #echo 2 dagen
elif [ $UURVDAG -lt 6 ]
  then
    if [ $WEEKDAG -gt 1 ]
      then DELTA=86400
      #echo 1 dag
      else DELTA=259200
      #echo 3 dagen
    fi
fi


#DATUMVANDAAG is latest weekday
DATUMVANDAAG=`date  -D %s -d $(( $(date +%s) - $DELTA )) +"%Y%m%d"`
#echo $DISPLAY >> /home/user/dldepapers/download.log
#DISPLAY=:0
#echo $DISPLAY >> /home/user/dldepapers/download.log
#echo /home/user/MyDocs/Downloads/ZH_$DATUMVANDAAG.pdf

if [ -e /home/user/MyDocs/Downloads/ZH_$DATUMVANDAAG.pdf ]
then
 echo "file /home/user/MyDocs/Downloads/ZH_$DATUMVANDAAG.pdf exists" >> /home/user/dldepapers/download.log
 /usr/bin/office-tools /home/user/MyDocs/Downloads/ZH_$DATUMVANDAAG.pdf
else
 echo "file /home/user/MyDocs/Downloads/ZH_$DATUMVANDAAG.pdf doesn't exist, trying to download" >> /home/user/dldepapers/download.log
 cd /home/user/MyDocs/Downloads
 grob http://www.20min.ch/printpdf/ZH_$DATUMVANDAAG.pdf
fi

This one will actually download the latest version; it takes into account that the new edition will be posted at 6am, before that it will get the previous edition - so before Monday 6pm, it will fetch the one from Friday.
If the file already exists, it will be opened in the pdf viewer.


Then I created an icon entry for the desktop:
Code:

cat /home/user/.local/share/applications/20min_pdf_dl.desktop


[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Name=20min PDF
Comment=20min downloader script
Exec=/bin/sh /home/user/dldepapers/getlatest20minbutton.sh
Icon=/home/user/dldepapers/20minicon.png

plus naturally an icon, as you can see.


Now, same thing for Blick am Abend:

Code:

cat getlatestblickamabend.sh
#!/bin/sh
DELTA=0
WEEKDAG=`date +%u`
UURVDAG=`date +%H`
if [ $WEEKDAG == 6 ]
  then DELTA=86400
  #echo 1 dag
elif [ $WEEKDAG == 7 ]
  then DELTA=172800
  #echo 2 dagen
elif [ $UURVDAG -lt 15 ]
  then
    if [ $WEEKDAG -gt 1 ]
      then DELTA=86400
      #echo 1 dag
      else DELTA=259200
      #echo 3 dagen
    fi
fi

DATUMVANBAA=`date  -D %s -d $(( $(date +%s) - $DELTA )) +"%Y-%m-%d"`
#echo $DATUMVANBAA
if [ -e /home/user/MyDocs/Downloads/"$DATUMVANBAA"_zh.pdf ]
then
 echo "file /home/user/MyDocs/Downloads/"$DATUMVANBAA"_zh.pdf exists already, opening..." >> /home/user/dldepapers/download.log
 /usr/bin/office-tools /home/user/MyDocs/Downloads/"$DATUMVANBAA"_zh.pdf
else
 echo "file /home/user/MyDocs/Downloads/"$DATUMVANBAA"_zh.pdf doesn't exist, trying to download" >> /home/user/dldepapers/download.log
grob http://php.blick.ch/baa_epaper/"$DATUMVANBAA"_zh.pdf
fi

This one is published at 4pm, so before 4pm, it fetches or opens the previous edition.

Desktop icon entry:
Code:

cat /home/user/.local/share/applications/baa_pdf_dl.desktop


[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Name=BaA PDF
Comment=BaA downloader script
Exec=/bin/sh /home/user/dldepapers/getlatestblickamabend.sh
Icon=/home/user/dldepapers/blickamabend_icon.png


In case you didn't guess yet - indeed, no data contract,.. our provider charges CHF0.49 / 10kB.
But among each other my wife and I get unlimited free calls and free sms-es, at a monthly rate of CHF 0 (yes, zero), so we won't be changing to a different plan with data anytime soon...

pycage 2012-02-04 17:54

Re: online newspaper download scripts
 
Speaking about 20 Minuten... there's an official app for this newspaper for free in the Nokia store.

aRTee 2012-02-05 09:36

Re: online newspaper download scripts
 
You mean this one:
http://store.ovi.com/content/249828
?

That can hardly be called an app... seems to be links only.
Besides, even if an app will allow for offline reading, you still have to remember to launch it within range of your wifi, unless you have a data plan, in which case you don't care for an app, because you can just read the online version...


All times are GMT. The time now is 12:11.

vBulletin® Version 3.8.8