PGP SIGNED MESSAGE
Hash: SHA1
I am experimenting with writing a program that does not rely on LineMode
to send and receive data. The following procedure is attached to a
server component:
TriggerWriteDebug is my own event, which at this time only outputs text
to a TMemo.
Also, I plan on changing the code to except Network Byte numeric
data, but for now I want to get this out of the way. :-)
procedure (Sender: T; ErrCode: Word);
var
I: Integer;
Buff: Array[0_MAX_BUFF - 1] of Byte;
begin
TriggerWriteDebug('DataAvailable() enter');
if (TMySockServerClient(Sender).State <wsClosed) then
begin
I := ReadFully(Sender, @Buff, 5);
end;
TriggerWriteDebug('DataAvailable() exit.');
end;
ReadFully is another procedure I have written in hopes that I can then
read an entire string, word, integer, what have you and read until I
gather up enough data to continue (since I'm not using LineMode).
function TMySockServer.ReadFully(Sender: T; Data: Pointer;
BytesExpected: Integer): Integer;
var
Buffer: PChar;
: Integer;
Received: Integer;
begin
:= 1;
GetMem(Buffer, BytesExpected);
while (BytesExpected 0) do
begin
Received := TMySockServerClient(Sender).Receive(@Buffer[],
BytesExpected);
if (Received 0) then
begin
:= + Received;
BytesExpected := BytesExpected - Received;
end;
end;
TriggerWriteDebug(Buffer);
Result := ;
end;
I have a couple of questions:
When I output Buffer in the last lines of ReadFully(), if I have a
client that sends "blah!" (which is five characters), the Memo has a
non-displayable character (but shows up on the Memo as a black block)
and then 'blah!'. So I'm not sure if I'm grabbing the string correctly.
I hope with minimal changes that this procedure might also grab numbers
without error, too.
When my test client disconnects from the server, the server enters its
ClientDataAvailable event and does not exit, which is why I have a "if
(TMySockServerClient(Sender).State <wsClosed)" line in my
ClientDataAvailable - I was hoping that I could detect whether or not
the socket is still connected and only then try to read data. As of
right now, it stays in DataAvailable and my server goes into an infinite
loop. :(
I realize the advantages of a LineMode = TRUE. I may just switch, but
at the current time I'm just trying to learn more about sockets and
building custom protocols in general.
Does anyone have any advice for me to make these routines operational or
more robust?
Cheers,
Wes
PGP SIGNATURE
Version: GnuPG v1.4.0 (MingW32)
JnYB2ex9LdhMnuX+aaNszX0=
=KZhv
PGP SIGNATURE

ReadFully is another procedure I have written in hopes that I can then
read an entire string, word, integer, what have you and read until I
gather up enough data to continue (since I'm not using LineMode).

Your ReadFully won't work. You must think event driven. When DataAvailable is triggered, you must
read data but you can't loop waiting for so much data to come. If you have not received enough data,
just get thereceived data into a buffer and return from the event handler. When the next
DataAvailable is triggered, you append the next data read into your buffer, until you get enough
data. Basically, that's what LineMode is doing, until it has received a complete line.

In addition, you have a memory leak: you allocate memory and never free it.

PGP SIGNED MESSAGE
Hash: SHA1

Francois Piette wrote:
Your ReadFully won't work. You must think event driven. When DataAvailable is triggered, you must
read data but you can't loop waiting for so much data to come. If you have not received enough data,
just get thereceived data into a buffer and return from the event handler. When the next
DataAvailable is triggered, you append the next data read into your buffer, until you get enough
data. Basically, that's what LineMode is doing, until it has received a complete line.

K. I understand what you're saying, which for me is a definite step
up. That being said, when would I actually act on the data that I have
received from the socket? If I have LineMode off, that means I have to
either 1) look for a terminator or 2) accept <xmany bytes and then act
on the information I have so far, correct?

Again, as I've seen in archive here before, what happens if my
information gets processed in two batches (packets)? When do I decide
K, enough is enough, and continue on processing? Is TWSocket
neccessarily fit for binary-type protocols like Napster/ICQ/WASTE/etc
with variable-length packets?

In addition, you have a memory leak: you allocate memory and never free it.

Yeah, I noticed that. Thanks for bringing it to my attention. That's
like the 12th Commandment or something (the 11th being "Thou shalt not
divide by zero!") :-)

Cheers,
Wesley

PGP SIGNATURE
Version: GnuPG v1.4.1 (MingW32)

J+PTuWNJUGU2Fr81e+YwSoI=
=jsDA
PGP SIGNATURE

K. I understand what you're saying, which for me is a definite step
up. That being said, when would I actually act on the data that I have
received from the socket? If I have LineMode off, that means I have to
either 1) look for a terminator or 2) accept <xmany bytes and then act
on the information I have so far, correct?

Correct. You can select either way. In anycase, don't forget that you may
receive more data that what you want. course you have to keep the extra
data into your buffer. For example, even if you send an integer (4 bytes) as
length and then send the actual data in a separate send, you may well
receive your count and your actual data in a single packet and a single
DataAvailable event.

Again, as I've seen in archive here before, what happens if my
information gets processed in two batches (packets)? When do I decide
K, enough is enough, and continue on processing? Is TWSocket
neccessarily fit for binary-type protocols like Napster/ICQ/WASTE/etc
with variable-length packets?

Yes of course. What you talk about is not TWSocket specific. It is just that
TCP/IP work like that. Specially, TCP is a stream oriented protocol. It is
suitable for transmitting any kind of data using any kind of TCP/IP based
protocol.

Hello Wesley,

If I have LineMode off, that means I have to
either 1) look for a terminator or 2) accept <xmany bytes and then act
on the information I have so far, correct?

yes

Again, as I've seen in archive here before, what happens if my
information gets processed in two batches (packets)? When do I decide
K, enough is enough, and continue on processing? Is TWSocket
neccessarily fit for binary-type protocols like Napster/ICQ/WASTE/etc
with variable-length packets?

Sorry I dont understeand what you asking here. You can send binary data
of course but if you wants to use LineMode then you have to escape some
bytes, otherwise you have to send some length byte/word/dword.

You find on my site some additional demo file witch may help you.

Rgds, Wilfried
http://www.mestdagh.biz

PGP SIGNED MESSAGE
Hash: SHA1

Wilfried Mestdagh wrote:
>>Again, as I've seen in archive here before, what happens if my
>>information gets processed in two batches (packets)? When do I decide
>>K, enough is enough, and continue on processing? Is TWSocket
>>neccessarily fit for binary-type protocols like Napster/ICQ/WASTE/etc
>>with variable-length packets?

Sorry I dont understeand what you asking here. You can send binary data
of course but if you wants to use LineMode then you have to escape some
bytes, otherwise you have to send some length byte/word/dword.

I guess what I'm trying to say is that with a conventional program, you
can initiate a send, then for example if you receive a certain packet
type, read more for one operation (say receiving a file listing after
asking the server for it) and then keep your program going.

For me it is difficult to understand how to "packetize" a TCP stream and
rebuild packets on the receiving side, and only read what I think is
neccessary for a certain programmer-defined packet type (and then act on
them) when not using LineMode and using the *event-driven model* ICS
employs.

It's easy with LineMode on - ICS only forwards the data on when it
receives a delimeter. But when it's off, it seems as though anything
other than file sends and receives (because you don't really act on the
file stream, you just send or recieve its bytes) proves to be a
difficult task.

Can you share some insight?

Cheers,
Wes
PGP SIGNATURE
Version: GnuPG v1.4.0 (MingW32)

6EHgR/41vE9lCLit6LQY2yg=
=Hetw
PGP SIGNATURE

It's easy with LineMode on - ICS only forwards the data on when it
receives a delimeter. But when it's off, it seems as though anything
other than file sends and receives (because you don't really act on the
file stream, you just send or recieve its bytes) proves to be a
difficult task.

If not using a delimiter, you must use a byte count. In any case, the receiver need to know how much
data has been sent by the sender. TCP doesn't preserve boundaries: it send all data in the correct
order, without error and only once. But it can split or merge data according to what is needed by
the network layer.

Most protocols use line mode (SMTP, PP3, NNTP, ) some use both : FTP and HTTP. FTP use two TCP
connections: one for commands and one for data. command stream, it use line mode. data stream,
it just send data. End of data is found because the stream is simply closed. HTTP use a single TCP
stream, begin with line mode in his header and then switch to byte count for the document. Byte
count is given in one of the header lines.

So what do you have to do ? You have to design your own protocol, using line mode or not, it is not
very important. What is important is that at any moment one side know what the other is trying to
do.

I understood that you don't want to use line mode. Perfect for me. Use a byte count. You can, for
example build each of your message with 4 byte being a 32 bit integer that gives the number of data
bytes that follow. Send is trivial. Receiving require a buffer where you append everything you
receive. Each time you received something (DataAvailable event, then call Receive method), you
check how many bytes you have received. If less than 4, you just do nothing more. If more than 4,
then you received your count. You compare the count with the actual data received. If you have
enough data, you trigger your own event, such as MessageReceived, passing the count of bytes and
the pointer to the first byte into your buffer (You can also copy the data but if will be slower).
When the event handler returns, you remove the count and the message data from the buffer, keeping
the rest for the next round.

Hello Wes,

In addition to the advice of Francois is what I often do if designing a
protocol that is both ascii data (for commands) and binary data packets,
is to preceide the packet with a length in hexadecimal form (byte, word
or dword). Wy ? Because if most of the data is ascii command then it
make the readability better if you log the data for debugging purposes.

Rgds, Wilfried
http://www.mestdagh.biz

Thursday, June 2, 2005, 12:37, Francois Piette wrote:

>It's easy with LineMode on - ICS only forwards the data on when it
>receives a delimeter. But when it's off, it seems as though anything
>other than file sends and receives (because you don't really act on the
>file stream, you just send or recieve its bytes) proves to be a
>difficult task.

If not using a delimiter, you must use a byte count. In any case,
the receiver need to know how much
data has been sent by the sender. TCP doesn't preserve boundaries: it send all data in the correct
order, without error and only once. But it can split or merge data according to what is needed by
the network layer.

Most protocols use line mode (SMTP, PP3, NNTP, ) some use both : FTP and HTTP. FTP use two TCP
connections: one for commands and one for data. command stream, it use line mode. data stream,
it just send data. End of data is found because the stream is
simply closed. HTTP use a single TCP
stream, begin with line mode in his header and then switch to byte count for the document. Byte
count is given in one of the header lines.

So what do you have to do ? You have to design your own protocol, using line mode or not, it is not
very important. What is important is that at any moment one side know what the other is trying to
do.

I understood that you don't want to use line mode. Perfect for me. Use a byte count. You can, for
example build each of your message with 4 byte being a 32 bit
integer that gives the number of data
bytes that follow. Send is trivial. Receiving require a buffer where you append everything you
receive. Each time you received something (DataAvailable event, then call Receive method), you
check how many bytes you have received. If less than 4, you just do nothing more. If more than 4,
then you received your count. You compare the count with the actual data received. If you have
enough data, you trigger your own event, such as MessageReceived, passing the count of bytes and
the pointer to the first byte into your buffer (You can also copy the data but if will be slower).
When the event handler returns, you remove the count and the
message data from the buffer, keeping
the rest for the next round.