Querying external websites in FREE apps
 

I havent found any definitive answer for this.

is it illegal to distribute *free apps* that display html content ( without modifying ) by querying external websites ?

I know that some websites provide apis's to access their content, but the majority dont have it , so to create the query in your app and get the data to display it on the phone using a *free app*, would this also count as web scraping and be considered illegal ?
Perhaps some of those websites rely on advertising for revenue so may be case to case also ?

I think all of us creating apps must be aware of this so we dont end up in legal issues, and also hosting on maemo.org would only highten the problem.
are there any rules imposed by maemo.org on this ?

thanks.

Re: Querying external websites in FREE apps
 

So you are asking if web browsers are illegal? :)

I don't see how it could be illegal, as it would be technically no different that viewing the content with a web browser with an ad blocker...

Re: Querying external websites in FREE apps
 

well its a little different here from a normal browser ;), on a browser you get to see the actual website in its entirety, including the ad banners on a website (?) and links to their sponsors blah blah. ( inspite of the ad blocker i suppose ? )
Sorry maybe I wasnt clear when I said I dont modify the html data.

In apps, we would not like to display the entire content as it would display on a normal browser with ad blocker , so we extract the relevant data from the response html and display what we want
( we arent changing data but just the format thats what i had meant by not modifying data )
hope that clarifies :)

Re: Querying external websites in FREE apps
 

I don't see how it would be illegal at all.A few Apps on Extras pull data from external websites.

Re: Querying external websites in FREE apps
 

could you tell me a few examples (app and website they access) to look up please.
cheers.

Re: Querying external websites in FREE apps
 

omweather, for example

Re: Querying external websites in FREE apps
 

Thanks.

But, I just went through their source code.
seems like THIS is one of the feeds they use and its free and available to public as per the weather.com website ( so this is an exception since its explicitly provided by the data source )

" Weather XML Data Feed
Now you can include weather data from The Weather Channel® in your website, desktop application, or widget in return for proper attribution and five (5) required links back to www.weather.com. Sign up now – It's FREE"

I agree websites that provide developers with api's or feed are legal but not many do as I stated in my initial post, I was referring to data from those websites that dont provide such a feed. What about them ?

cheers

Re: Querying external websites in FREE apps
 

some websites also explicity state their intellectual rights like this

Re: Querying external websites in FREE apps
 

That really would be on a site by site basis and per their use agreement (If they have one) Its not illegal. Maemo OS and licenses don't seem to care.

EDIT: You could also request if you are allowed to access the website via an App you are writing to any websites that talk about consent

Re: Querying external websites in FREE apps
 

If you do request access, I would suggest spending some time formulating the question. Asking "Can I rip your website?" vs. something like "I really like your website. I would like to offer this information to mobile users... What kind of attribution and linking would be required? etc."

Re: Querying external websites in FREE apps
 

LOL, sure :D

Re: Querying external websites in FREE apps
 

I was actually thinking the same thing just after I hit reply ;)

Re: Querying external websites in FREE apps
 

So, how does it differ then from FF + AdBlock Plus + Greasemonkey then? :cool:

It's quite of a gray area, but unless the target website specifically condemns such usage of their web site, I think you are pretty much in the clear :)

Re: Querying external websites in FREE apps
 

Im assuming the same here too actually, that if they dont state any such intellectual property rights on their page then its safe to query their website with our app created queries.:p

Since this is a grey area , hence the thread.
My intention of raising a thread was to be sure we know what we are doing when creating such apps, was hoping to get some more concrete information actually, maybe there are people around who may have created apps like ive described and/or have done their homework for this to share it here with us.
examples would definetely help.

Re: Querying external websites in FREE apps
 

Well, you can always use the old argument of guns not killing people - people kill people (tho the gun certainly helps ;) ), that has been used as an excuse of many developers of somewhat questionable applications.

Your app won't be, ehm, stealing data from any site - users will. And good luck to the authorities for chasing individual users, if that were possible, piracy wouldn't exist. :cool:

Re: Querying external websites in FREE apps
 

It is just like everything else. What is dubious and should be illegal is receiving monetary compensation or consideration of any value from the fruits of some one else's labor or established intellectual property.

HTML is the original "Free and Open Source" code and should remain so. Many have "scraped" or linked directly to other peoples content in order to by-pass un-needed information or to format the information for a different platform.
I do it all the time and store my efforts locally on my tablet and/or N900.


Now if I charged money for these pages or apps which use this content or included the code in a page on my website that I receive revenue from different ads and such, I should pay the producer for his efforts or expect to be held liable.

(This of course is a bit simplified for the sake of this discussion. Other liabilities exist such as altering trademarks and what not.)

If someone was using my data there are ways to obscure it, but others will always find a way to use it. It is easy however to track this use, and lawyers who are willing to serve, are piled up like cord wood outside the doors. ;)

This system has worked since the Romans ruled the world and as far as I'm concerned it is not broken. HTML should always remain open.

One of my peeves when the iPhone first came out was in it's intent to upset this order by requiring content to be specifically formatted for its platform. This model of theirs has changed since its introduction because users demanded more... and that's a good thing.

On the plus side though, Apples model has kept legal fracases over content behind the doors of their app store where makers of the many different iFart apps can argue over who was first, or who's fart is the nastiest.

Meh...
I don't care, and have been happily pulling and using content from sites like Paul's Weather Page since I first cobbled together a windows socket back in the day . :p

Edited Update:
When I lived in the Bay Area I first started to go to Paul's page in order to quickly find needed information like "Will my campsite be flooded this week-end." or just knowing what clothes to pack for a short trip. Viewing the source code from his page back then and using it to experiment with variations on my own pages is what taught me how to use HTML in the first place.

For those who can remember back in the day, most information was only available via FTP so pulling and posting via HTML was a service for those who didn't have, or were not inclined to access it via Archie, Gopher, Veronica and Jughead . Also, there were only about 3,500 pages available then to use as a reference. Some of us even met up on the weekend in places like the parking lot of Fry's in Fremont, or the Point in Saucalito to swap code saved to floppies or written down on pieces of paper. :eek: It was truly a small world then.

This thread prompted some reminiscing (thank you). As a result I was just cruisin' his site where I came up with the following posting when I clicked on one of his links.

Unfortunately that^ was posted almost 3 years ago and sadly it is often the outcome when people become overly protective of even publicly sponsored data.
A good idea and useful service becomes stifled and the common man ends up with a soggy sleeping bag wearing the wrong dang shoes. :D

Re: Querying external websites in FREE apps
 

LOL, that post was good read YoDude ! ;)

just called up a website that I wanted to access through an app.
I mentioned all the kind words :D and the max that I could credit them with is a free mention of them in my free app :)

they said they would look into it and get back.

somehow i feel better now that i called them, this got me so hassled ;) , though I know its not right to keep data unshareable which is accessed by the public for the public .