put URL... does not work. Why? Please help me! (='.'=)

[Mariasole]

Hello!
I apologize again for my horrible english!
I'm studying LiveCode and I'm excited... i'm a beginner, an absolute beginner!
Right now I'm doing some tests on downloading web pages. I noticed a strange behavior.
Here is the code. (There is a button and 3 fields)

Code: Select all

on mouseUp
               
put url "http://www.ilfattoquotidiano.it" into field "fHtml"

end mouseUp

LiveCode can not download the page correctly!


I built a routine more complex to understand what does not work ... here it is:

Code: Select all

on mouseUp
               
put url "http://www.ilfattoquotidiano.it" into myData --- put your other url here for test! ;)
          
            
-- set Header (it's an example of User Agent, I have tried many other... win, mac, linux etc...!
set the HTTPHeaders to "User-Agent: Firefox 20/Linux: Mozilla/5.0 (X11; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0"  \
   & return & "Connection: Keep-Alive"
            
            
            
put uniencode(myData,"UTF8") into myData

-- put in my Human Viewer field (fHtml) ;)
set the unicodeText of fld "fHtml" to myData

 -- view Header in field fHeader
put  libUrlLastRHHeaders() into field "fHeader"
   
 -- 5. view the UserAgent in field fUserAgent
put libURLLastHTTPHeaders() into field "fUserAgent"

end mouseUp

TEST progress... getting page...

http://www.cnn.com OK!!!!

http://forums.runrev.com/ OK!!!! W LIVE CODE!!!! W THE LIVE CODE FORUM!!!!

http://www.huffingtonpost.com OK!!!

http://www.ilfattoquotidiano.it NOOOOOOOOO!!!!!!!!! why?


This website, however, shows "squares" and I can not see the code in the correct way!


where am I wrong?

Maybe it's a stupid thing ... but for me it is a major obstacle! Thanks to those who want to help me!

Baci!!!!

Mariasole (='.'=)

PS: I use WinXP and LiveCode Community 6.0

[BvG]

Some HTML servers offer the ability to GZIP compress their sites, to reduce bandwidth. They ought to only do it if the browser (in this case the LiveCode script) accepts compressed data in the headers. It seems that this site always sends compressed data, no matter what. To actually read it, just do :

Code: Select all

put decompress(url "http://www.ilfattoquotidiano.it")

[Mariasole]

Thank you very much BvG!!!!!

I tried your solution and it works!!

But if I have a list of URLs, compressed and uncompressed, to download, how can I know their status "before"?
In fact, if I "decompress" a url where the server is not "compressed", Live Code gives me a error!
for example:

Code: Select all

put decompress(url "http://forums.runrev.com") into mydata

button "download!": execution error at line 4 (decompress: string is not compressed data), char 1




I thought this:

Code: Select all

   put "http://www.ilfattoquotidiano.it" into vURL --- compressed site
  -- put "http://forums.runrev.com" into vURL --- not compressed site
   
   put URL vUrl into myData
   
  if myData contain compress data THEN put decompress(url vUrl) into field "Compressed" 
  if myData NOT contain compress data THEN put vUrl into field "NOT Compressed"

But unfortunately I do not know how to write it ... would you give me a hand?
Thank you! And may God bless you!

baci!
Mariasole (='.'=)

[FourthWorld]

In many cases, such as with the decompress function, the engine will throw an error if it encounters data that isn't in the expected format. In such cases you'll just have to try it - using the "try" control structure:

Code: Select all

put url "http://forums.runrev.com" into myData
-- First, check for errors in the download:
if the result is not empty then
   answer the result
   exit to top
end if 
-- Now try to decompress the data:
try
  put decompress(myData) into myData
catch tErr -- this param contains a string containing error lookup info which can sometimes be useful, 
   -- but we won't use it here since we already know the download was successful; if decompress
   -- fails we're probably just dealing with uncompressed data, so we do nothing special to handle
   -- the decompression error.
end try
DoSomethingWith myData

See the Dictionary entry for "try" for details.

[sturgis]

if you "get" the url, then look at what liburllastRHheaders() returns, look for "content-encoding: and it will say "gzip" if its compressed with gzip.

So you can do something like this:

Code: Select all

put (url "http://www.ilfattoquotidiano.it") into tData -- grab the data
--EDIT, Seems to need a "wait with messages" line to make it reliable so..
wait 0 with messages 

if liburllastRHHeaders() contains "gzip" then -- look for gzip as part of the headers
put decompress(tData) into tResult -- if gzip was found, decompress
else
put tData into tResult -- otherwise just use the data
end if
put tResult -- puts the result into msg box

[FourthWorld]

if you "get" the url, then look at what liburllastRHheaders() returns, look for "content-encoding: and it will say "gzip" if its compressed with gzip.

Even better solution. Good work, Sturgis.

[Mariasole]

Thanks FourthWorld and Sturgis for your kindness and expertise!
I'm testing both solutions (both work!).
But the fact that there are at least two ways of solving the problem shows me that Live Code is not a toy, but a real programming language "mature" for every need! [enthusiasm of the neophyte? ... )
Now I will study the structure "try", and if the header of the webpages obbligatory declared "gzip" in string when it "pass" a compressed html.
Thanks again Sturgis and FourthWorld, sorry for my horrifying english and God bless you!

Mariasole
(='.'=)

"But we're absolute beginners
But if my love is your love
We're certain to succeed"

[FourthWorld]

Thanks FourthWorld and Sturgis for your kindness and expertise!
I'm testing both solutions (both work!).
But the fact that there are at least two ways of solving the problem shows me that Live Code is not a toy, but a real programming language "mature" for every need! [enthusiasm of the neophyte? ... )
Now I will study the structure "try", and if the header of the webpages obbligatory declared "gzip" in string when it "pass" a compressed html.
Thanks again Sturgis and FourthWorld, sorry for my horrifying english and God bless you!

Mariasole
(='.'=)

"But we're absolute beginners
But if my love is your love
We're certain to succeed"

All of us are beginners in at least one thing or another. There's always something new to learn, and this forum is a great place for such exchanges.

The "try" construct is useful for many things, but in this case the advantage of checking the headers as Sturgis suggested is even better because it reduces the range of possible sources for errors to check for if anything goes wrong.

One thing you'll find as you ship more apps is that in most apps only about half the code is getting the app to do what you want, and the rest is handling errors when it doesn't. Good apps check for as many specific errors as practical, and attempt to provide some sort of graceful degradation when things don't go as planned.

"Try" is a good way to catch many things, and the error info contained in the tErr argument I noted in my example can be parsed to determine specifics about the error if needed. But Sturgis' example is, IMO, reflective of a better style, using native language options to explicitly check the nature of the data.

There's a point at which even error-checking reaches diminishing returns, in which the effort expended to trap for every possible thing is greater than the likelihood of those things actually going wrong. But with Internet connectivity an app is relying on a great many systems beyond your control (the OS, the router, the server, and all the parts in between), so it's a good area for putting in a little extra effort to determine specifics when errors happen.

As for your English, you needn't worry. One of the great things about the language is that many of its speakers are Americans, who often show surprisingly little interest in using it well. This provides a great opportunity for non-native English speakers like yourself, as your English is better than that of many Americans.

[Mariasole]

Thank you Richard for your wonderful lesson! I am so thankful!
The LiveCode Forum is a great help for beginners like me.
A place where the gurus (and you're a super guru!) are put at the disposal of newbie friendly!
Thank you for your help, your wisdom and your understanding ... and irony!

Baci!!!

Mariasole
[='.'=]