Page 1 of 3
Timing over socket connection
Posted: Thu Dec 01, 2022 4:33 pm
by trevix
- PCa is connected over local network and sockets to PCb.
- PCa send a message to PCB that must do something with the received message (make take 1 or 2 seconds)
- Upon completion, PCB send a CONF message to PCa, just to say that everything went well or to provide some data.
- After receiving the CONF msg, PCa can go on with its tasks.
In order to accomplish this, I scripted what you can find below. But I am having problems.
Is it the wrong way to do it or my scripts have bugs?
Thanks.
Code: Select all
function SendToServer
write tLenght & comma & TheMessage to socket gSocket
wait 500 milliseconds with messages
--"ClientMessageReceived" read the size (first item) and do other reading pass
read from socket gSocket for 1 items with message "ClientMessageReceived"
--the idea here is that "ClientMessageReceived" has time to set the sRePlayReceived flag
repeat 9
if sRePlayReceived then
put false into sRePlayReceived
return true
end if
wait 500 milliseconds with messages
end repeat
return false
end SendToServer
Re: Timing over socket connection
Posted: Fri Dec 02, 2022 4:43 am
by FourthWorld
trevix wrote: ↑Thu Dec 01, 2022 4:33 pm
But I am having problems.
What problems are you having?
Re: Timing over socket connection
Posted: Fri Dec 02, 2022 9:42 am
by trevix
Sometime, while on a "wait for 500 milliseconds with messages" I am not sure that the "read from socket" happens.
As I understand, LC only do things serially.
Thus, what happens if the "read" takes more time then the 500 milliseconds of the "send" loop? (the "read" also is on a loop, given the eventual size of the content).
Essentially there is a main wait loop (send) and a secondary wait loop (read) that I guess runs only on the wait moments of the main loop.
Or not?
Far out...that's why I think I may have it all wrong...
Re: Timing over socket connection
Posted: Fri Dec 02, 2022 11:32 am
by trevix
Also.. is this the only way to clear the socket buffer, while keeping the connection open?
Code: Select all
read from socket TheSocket until "*" --*= some non existent char
Re: Timing over socket connection
Posted: Thu Dec 08, 2022 6:00 am
by Cairoo
Hi Trevix,
trevix wrote: ↑Fri Dec 02, 2022 9:42 am
Sometime, while on a "wait for 500 milliseconds with messages" I am not sure that the "read from socket" happens. ...
Looking at your code, you're using a callback message to notify you when the server's response has arrived. If the callback message hasn't been received by the time the 500 milliseconds wait is over, then you can be sure that no data has arrived yet.
trevix wrote: ↑Fri Dec 02, 2022 9:42 am
... Thus, what happens if the "read" takes more time then the 500 milliseconds of the "send" loop? (the "read" also is on a loop, given the eventual size of the content). ...
You gave it 9 intervals of 500 milliseconds each in which to finish reading the server's response.
If your ClientMessageReceived handler does not finish reading the data in that time, your SendToServer function would return false.
So if you really want the SendToServer function to return true only if the server's response was completely read within 4500 milliseconds, then your SendToServer function is good to go.
trevix wrote: ↑Fri Dec 02, 2022 11:32 am
Also.. is this the only way to clear the socket buffer, while keeping the connection open?
Code: Select all
read from socket TheSocket until "*" --*= some non existent char
The read command clears the socket buffer unless told to read only a specified amount of chunks or to read until it encounters a specified string. After clearing the read buffer, the socket normally remains open until closed explicitly on either end or until the app quits.
If the problems you're having only occurs with large amounts of data, it may well be that your ClientMessageReceived handler takes longer than 4500 milliseconds to finish reading the data.
If you want your SendToServer function to return true as soon as the server's response arrives, i.e. even before the server's response has been fully read, then consider setting the sRePlayReceived flag at the start of your ClientMessageReceived handler.
Re: Timing over socket connection
Posted: Thu Dec 08, 2022 6:15 am
by Cairoo
trevix wrote: ↑Fri Dec 02, 2022 9:42 am
... Essentially there is a main wait loop (send) and a secondary wait loop (read) that I guess runs only on the wait moments of the main loop.
Or not? ...
The secondary read loop should happen in the ClientMessageReceived handler and unless something unusual happens, would complete independently from the main send loop. Your main send loop basically waits for your secondary read loop to complete but may not be giving it enough time if the server's response is large.
Re: Timing over socket connection
Posted: Thu Dec 08, 2022 11:49 am
by trevix
Hello and thanks for answering.
Everything you wrote is correct and this is how my script works.
Having the server send a confirmation before the read, or even removing the confirmation doesn't solve my problem when the size of the returned package is over...I would say around 120k.
Under that amount my client script reads every that has been sent correctly.
Over that amount, the received data are somehow scrambled.
My script is a little more complex of what I presented as an example, but essentially the client receives:
- a number to specify how much to read && a short command && comma
- a base64 encoded data,that gets read on a loop for 72 chars
Check the script below.
As I said, when the payload is too big, the data received is, for example:
- first item= 126 12 DOTHIS --correct
- empty -- not correct. as any subsequent reading
then if I do again a send/read, the client receives part of the binary, followed by & DOTHIS -- not correct
There must be a problem on socket reading big chunks.
The reading is incorrect for big chunks even if instead of using this method, I do a "read until char XX"
Code: Select all
if pMsg is empty then
read from socket p_Socket for 1 items with message "chatServerMessageReceived_new"
exit chatServerMessageReceived_new
else
put word 1 of pMsg into tNum --number of 72 chars chunks
put word 2 of item 1 of pMsg into tSubNum --the left over number of chars
if tNum is an integer then
put word 3 of item 1 of pMsg into tCommand
repeat tNum
read from socket p_Socket for 72
if it is empty then exit repeat --on big payload this is empty
put it after tRD
end repeat
if tSubNum <> 0 then --read leftovers
read from socket p_Socket for tSubNum
put it after tRD
end if
put tRD into pMsg
else
put item 1 of pMsg into tCommand
end if
----
As for clearying the socket buffer, the problem is that read until "NotExistingChar" takes forever, like10 seconds. There should be a command for fast doing this.
Re: Timing over socket connection
Posted: Sat Dec 10, 2022 2:12 pm
by Cairoo
Trevix
trevix wrote: ↑Thu Dec 08, 2022 11:49 am
- a base64 encoded data,that gets read on a loop for 72 chars
Reading the base64encoded data in chunks of 72 chars in a loop would most definitely slow things down.
Maybe you could consider splitting the binary on the sending end into large but less-than 90K sized chunks and send each chunk as a base64encoded string, and on the receiving end just read the whole chunk and base64decode it?
- Gerrie
Re: Timing over socket connection
Posted: Sat Dec 10, 2022 8:07 pm
by Cairoo
I've attached a simple example of sending binary data in chunks over a socket.
I've used a similar approach with sending a header with every chunk, as you have.
Let me know if it helps.
- Gerrie
EDIT: Don't use the one attached here; see next post.
Re: Timing over socket connection
Posted: Sun Dec 11, 2022 7:55 am
by Cairoo
To my embarrassment, I haven't tested my sample stack properly. It worked well sending from my Windows PC to my Mac, but glitched out when sending from my Mac to my Windows PC.
I now think sending the chunks over in a repeat loop is the wrong way. I'm also learning.
Next I'll try a callback with the "write so socket" command and in the callback wait for a response from the server with another callback that sends the next chunk, and hopefully I can heal my reputation with a functional sample stack.
Re: Timing over socket connection
Posted: Sun Dec 11, 2022 10:04 am
by Cairoo
Cairoo wrote: ↑Sun Dec 11, 2022 7:55 am
... hopefully I can heal my reputation with a functional sample stack.
Attached below is the more functional stack.
- Gerrie
Re: Timing over socket connection
Posted: Tue Dec 20, 2022 4:28 pm
by trevix
Very well written Cairoo, thanks.
I noticed that you did not take in account socketTimeout. No need for it?
One more question:
I am from italy, where we use accented characters.
Loading a file to send, whose name is "Toni Carta Identità 2.jpg"
Code: Select all
the line urlEncode(textEncode(item -1 of it,"UTF-8")) into sUrlEncodedFileName -- return "Toni+Carta+Identita%CC%80+2.jpg"
--while
put files(tFolder,"detailed-utf8") into tFilesList --in the list the file is returned as Toni+Carta+Identit%C3%A0+2.jpg,856519,38521,1210322211,etc
This way no file is listed on tFileList.
Probably there is an easy way to solve this, but I wonder why the function "files()" must return an UrlEcoded name...
Regards
Re: Timing over socket connection
Posted: Tue Dec 20, 2022 4:55 pm
by trevix
I solved the second question like this:
Code: Select all
put item -1 of it into tName
...
-- get the file size
put files(tFolder,"detailed-utf8") into tFilesList
repeat for each line tLine in tFilesList
put textDecode(URLDecode(item 1 of tLine), "utf8") into tNameFromList
if tNameFromList = tName then --found
put tLine into tFilesList
exit repeat
end if
end repeat
put item 2 of tFilesList into tFileSize
Re: Timing over socket connection
Posted: Tue Dec 20, 2022 6:36 pm
by FourthWorld
trevix wrote: ↑Tue Dec 20, 2022 4:28 pm
I wonder why the function "files()" must return an UrlEcoded name...
The Mac Finder used to store folder metadata in a hidden file that use a return character at the end of its name, presumably to avoid being overwritten by any other file the user might create.
Since the value returned by "the files" is a return-delimited list, this of course made the list unusable.
Tho inconvenient, using urlEncode was a reasonably efficient solution to account for anything that might show up in a file name which could break the delimiting in the value returned from "the files".
Re: Timing over socket connection
Posted: Tue Dec 20, 2022 6:56 pm
by trevix
Thanks FourthWorld.
One more question for Cairoo, since you seem very confident with this stuff:
Since in your stack you made extensive use of local variables, to store ChunkNumer, TotalChunks, etc, what happens if 2 connected clients send each a different file to the server in the same time?
Do the recursions between the commands in the server keep each var istance separated from the others or the all thing must be rethinked?
Regards
Trevix