Problems with stdio on TCP (IPV4) sockets

[Lew Pitcher]

I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.

I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.

For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.

IF I

char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);

to extract the AMI version number that follows the text slash, my
fscanf() hangs.

BUT, if I

char buffer[1024], vrm[16];
fgets(buffer,sizeof buffer,fd);
sscanf(buffer,"Asterisk Call Manager/%s\r\n",vrm);

I manage to obtain the appropriate data. A dump of the
buffer array shows that I did indeed capture the introduction line
from the server.

So, why the difference in behaviour? Obviously, in the fscanf() version,
I've not set something up right. Any hints as to what I've done wrong
would be greatly appreciated.

Thanks in advance for your help

[Lew Pitcher]

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.

[snip]

Here's the code, and some trial runs

First, the code. It embodies three different ways of using stdio to read
the incoming stream.

With no compile options, it dumps (as hex values and as ASCII characters)
the first 29 octets read from the socket. This should (and does) represent
the introduction text sent by the server.

With the -DUSE_FGETS compile option, it uses fgets() to retrieve the first
text line from the server, and sscanf() to parse it. This works.

With the -DUSE_FSCANF compile option, it uses fscanf() to retrieve and
parse the first text line from the server. This hangs.

8<-------------------- main.c -------------------------------------->8
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <ctype.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>

/*
** NB: compile with
** -DUSE_FSCANF for fscanf() parse of 1st input line, or
** -DUSE_FGETS for fgets()/sscanf() parse of 1st input line
** otherwise code will use fgetc() to read the first 29 bytes of input
*/

int netOpen(char *host, char *port);

int main(void)
{
int status = 1;
int s_ami; /* base socket from netOpen(), fd for amiIn */

if ((s_ami = netOpen("localhost","5038")) >= 0)
{
FILE *amiIn;

if (amiIn = fdopen(s_ami,"r"))
{
#ifdef USE_FSCANF
{ /* use fscanf() to retrieve and parse a full line */
char vrm[16];

if (fscanf(amiIn,"Asterisk Call Manager/%s\r\n",vrm) == 1)
{
printf("Got introduction: AMI %s\n",vrm);
status = 1;
}
else puts("Cant retrieve VRM");
}
#else
#ifdef USE_FGETS
{ /* use fgets() to retrieve a full line, then sscanf() to parse it */
char buffer[256], vrm[16];

if (fgets(buffer,sizeof buffer,amiIn))
{
if (sscanf(buffer,"Asterisk Call Manager/%s\r\n",vrm) == 1)
{
printf("Got introduction: AMI %s\n",vrm);
status = 1;
}
else puts("Cant retrieve VRM");
}
else puts("Cant get introduction");
}
#else
{ /* naive dump of the first 29 bytes of input */
int byte;

for (int count = 0; count < 29; ++count)
{
if ((byte = fgetc(amiIn)) == EOF) break;
byte &= 0x7f;
printf("0x%02.2x %c\n",byte,((byte < 32)||(byte == 127))?' ':byte);
}
status = 1;
}
#endif
#endif
}
fclose(amiIn);
}
else fprintf(stderr,"Cant open AMI connection\n");

return status;
}


int netOpen(char *host, char *port)
{
int Socket;
struct addrinfo hint,
*address;

/* prepare the address */
memset(&hint,0,sizeof hint);
hint.ai_family = AF_INET;
hint.ai_socktype = SOCK_STREAM;
hint.ai_protocol = 0;
hint.ai_flags = 0;

if (getaddrinfo(host,port,&hint,&address) == 0)
{
if ((Socket = socket(AF_INET,SOCK_STREAM,0)) != -1)
{
if (connect(Socket,address->ai_addr,address->ai_addrlen) != 0)
{
close(Socket);
Socket = -1; /* connect() failure */
}
freeaddrinfo(address);
}
}
else Socket = -1; /* getaddrinfo() failure */

return Socket;
}

8<-------------------- main.c -------------------------------------->8

Now for the tests.

First off, a telnet session showing telnet's view of the server interaction.
23:43 $ telnet localhost 5038
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Asterisk Call Manager/7.0.0
^]
telnet> close
Connection closed.
23:43 $

The server sends "Asterisk Call Manager/7.0.0\r\n"

Next, the test program, compiled to dump the first 29 octets received
23:43 $ cc -o test1 main.c
23:44 $ ./test1
0x41 A
0x73 s
0x74 t
0x65 e
0x72 r
0x69 i
0x73 s
0x6b k
0x20
0x43 C
0x61 a
0x6c l
0x6c l
0x20
0x4d M
0x61 a
0x6e n
0x61 a
0x67 g
0x65 e
0x72 r
0x2f /
0x37 7
0x2e .
0x30 0
0x2e .
0x30 0
0x0d
0x0a

OK, so fgetc() can properly read the socket, and confirms the text including
the terminating \r\n combination.

Now, for the fgets()/sscanf() version:
23:44 $ cc -o test2 -DUSE_FGETS main.c
23:44 $ ./test2
Got introduction: AMI 7.0.0

That works as well.

Finally, the fscanf() version:
23:44 $ cc -o test3 -DUSE_FSCANF main.c
23:44 $ ./test3
^C
23:44 $ exit

Which hung. I let it sit for a few seconds, then killed it.

OK, you've seen the code, and you've seen the test results.

So, what did I do wrong in the fscanf() version? Any suggestions?

[Marcel Mueller]

Post by Lew Pitcher
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.

You do not want to use the buffered FILE API for bidirectional sockets
with message based protocols. This API tries to fill its buffer before
any other processing. And if the other endpoint does not send more data
so far it will block. This is a deadlock if the other endpoint will not
send more unless it gets your response.

Use the low level API (read, write etc.) for sockets. It will simply
return partial data if no more is available for now. Then you can decide
whether you expect more or not.

Typically the best is to use an appropriate library for the protocol you
need.


Marcel

[Lew Pitcher]

Post by Marcel Mueller

Post by Lew Pitcher
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.

You do not want to use the buffered FILE API for bidirectional sockets
with message based protocols.

[snip]

Post by Marcel Mueller
Typically the best is to use an appropriate library for the protocol you
need.

Thanks, Marcel, for the advice.

However, outside of a couple of out of maintenance, 3rd-party projects from
Asterisk users trying to do the same thing as I am, there's /no/ official
"library for the protocol" available, that I can find.

So, I either adapt one of those 3rd-party projects to my needs, or I roll
my own. And, since I need something to do, and something to learn, I decided
to roll my own protocol. :-)

Thanks again for the advice.

[Lawrence D'Oliveiro]

Post by Lew Pitcher
However, outside of a couple of out of maintenance, 3rd-party projects
from Asterisk users trying to do the same thing as I am, there's /no/
official "library for the protocol" available, that I can find.

My Python library, Seaskirt (link posted elsewhere) is the only one I know
of that covers AMI, AGI, Async-AGI, ARI and even the console interface. It
offers both synchronous and asynchronous (async/await) versions of all the
main API classes, and even allows for SSL/TLS connections where Asterisk
will accept these.

[Lew Pitcher]

Post by Lawrence D'Oliveiro

Post by Lew Pitcher
However, outside of a couple of out of maintenance, 3rd-party projects
from Asterisk users trying to do the same thing as I am, there's /no/
official "library for the protocol" available, that I can find.

My Python library, Seaskirt (link posted elsewhere) is the only one I know
of that covers AMI, AGI, Async-AGI, ARI and even the console interface. It
offers both synchronous and asynchronous (async/await) versions of all the
main API classes, and even allows for SSL/TLS connections where Asterisk
will accept these.

I will check it out. Thanks

[Lawrence D'Oliveiro]

For the line read, I decided to try stdio ...

Don’t. Your code is probably reading past the end of the response and
trying to get more.

Also, remember that Asterisk AMI terminates lines with CR/LF.

Sample Python code, from <https://gitlab.com/ldo/seaskirt/>, that
decodes responses properly:

while True :
endpos = self.buff.find(NL + NL)
if endpos >= 0 :
# got at least one complete response
resp = self.buff[:endpos + len(NL)] # include one NL at end
self.buff = self.buff[endpos + 2 * len(NL):]
response = {}
while True :
split, resp = resp.split(NL, 1)
if split != "" :
if split.endswith(":") :
keyword = split[:-1]
value = ""
else :
keyword, value = split.split(": ", 1)
#end if
if keyword in response :
response[keyword] += "\n" + value
else :
response[keyword] = value
#end if
if resp == "" :
break
#end if
#end while
break
#end if
# need more input
if self.EOF :
raise EOFError("Asterisk Manager connection EOF")
#end if
more = await self.sock.recv(IOBUFSIZE, timeout)
if more == None :
# timed out
break
self.buff += more.decode()
if len(more) == 0 :
self.EOF = True
#end if
#end while

[Lew Pitcher]

Post by Lawrence D'Oliveiro

For the line read, I decided to try stdio ...

Don’t. Your code is probably reading past the end of the response and
trying to get more.

Thanks, Lawrence

That's what I suspect, as well.

Post by Lawrence D'Oliveiro
Also, remember that Asterisk AMI terminates lines with CR/LF.

I know this all too well. But, thanks for the reminder

Post by Lawrence D'Oliveiro
Sample Python code, from <https://gitlab.com/ldo/seaskirt/>, that

[snip python code]

Thanks for that sample; I'll spend some time getting to know it.


Thanks again for your advice and help

[Tim Rentsch]

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets
programming and need some advice. [...]

I've done a fair amount of simple sockets programming. I wouldn't
call myself an expert but I'm certainly not a novice. I have
several bits of advice to offer.

Use read() and write() on any sockets. Avoid stdio.

Do the appropriate calls to make socket I/O non-blocking. At some
point you may want to get fancy and use select() or something
similar, but starting out it's probably good enough to do the
read()s and write()s in a loop, with a sleep for a hundredth of a
second in the loop so the program doesn't burn up cpu cycles.

Put all the calls to esoteric functions like getaddrinfo() in a
separate .c file, so the main body of code can be compiled with
options -std=c?? -pedantic, and only the one special .c file
needs all the special enabling to get non-standard functions.
(IIRC read() and write() do not need that kind of special
treatment, and can be mixed in with ISO conforming code with
no difficulty.)

Also, a general recommendation to avoid fscanf() altogether
and use sscanf() to do whatever scanning needs doing.

[Rainer Weikusat]

Post by Tim Rentsch

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets
programming and need some advice. [...]

I've done a fair amount of simple sockets programming. I wouldn't
call myself an expert but I'm certainly not a novice. I have
several bits of advice to offer.
Use read() and write() on any sockets. Avoid stdio.
Do the appropriate calls to make socket I/O non-blocking.

Non-blocking I/O is supposed to be used in situation where a
single-threaded program (or a single thread of a multi-threaded one)
needs to deal with data from multiple sources/ file descriptors. It's
not some kind of magic fairy dust for socket programming.

Post by Tim Rentsch
At some
point you may want to get fancy and use select() or something
similar, but starting out it's probably good enough to do the
read()s and write()s in a loop, with a sleep for a hundredth of a
second in the loop so the program doesn't burn up cpu cycles.

BSD terrorcoding at its finest. Never do anything properly, that's way
to much efforts! But hide the deficienies of your code well, they might
find you out otherwise!

If the program doesn't have anything else to do, it should block waiting
for something to do.

Post by Tim Rentsch
Put all the calls to esoteric functions like getaddrinfo() in a
separate .c file, so the main body of code can be compiled with
options -std=c?? -pedantic, and only the one special .c file
needs all the special enabling to get non-standard functions.

getaddrinfo is a standard function, just not an ISO-C standard function.

[Lew Pitcher]

Post by Tim Rentsch

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets
programming and need some advice. [...]

I've done a fair amount of simple sockets programming. I wouldn't
call myself an expert but I'm certainly not a novice. I have
several bits of advice to offer.

[snip very good advice]

Thanks, Tim, for the advice; I'll be sure to take it. It matches with
advice that I've seen from other sources, while I've been researching
this little problem.

The general impression that I've got is that stdio on sockets is...
tricky. (in my uniformed opinion, it /shouldn't/ be tricky; it is
very similar to stdio on a terminal device, in that there's no
lookahead, no unget, and must wait for it's input, just like a
terminal stream. But.... stdio on sockets /is/ tricky.

Looks like I'll have to use other means to access this socket stream.

Thanks again for the advice.

[Nicolas George]

Post by Lew Pitcher
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);

And wham the security flaw.

[Lew Pitcher]

Post by Nicolas George

Post by Lew Pitcher
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);

And wham the security flaw.

I suspect that you mean "buffer overflow" flaw.

Yes, in the wild, it would be.

But, this is an experiment (acknowledged as an experiment) under
laboratory conditions, where I /know/that the buffer is big enough
for the data (the string extracted will, in this case, /always/ be
"7.0.0", 6 octets, including the end-of-string character).

I will, however, fix this if and when my experiment grows into
something more useful.

Thanks for the catch.

[Nicolas George]

Post by Lew Pitcher
But, this is an experiment (acknowledged as an experiment) under
laboratory conditions

You posted it on Usenet.

[Lew Pitcher]

Post by Nicolas George

Post by Lew Pitcher
But, this is an experiment (acknowledged as an experiment) under
laboratory conditions

You posted it on Usenet.

Indeed, I did. And your point is?

[Nicolas George]

Post by Lew Pitcher
Indeed, I did. And your point is?

I am trying to determine if you belive that Usenet counts as “laboratory
conditions”.

[Rainer Weikusat]

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.

I did some tests with this with the built-in inetd echo server and the
fscanf, regardless of buffering mode the stream has been set to, simply
does another read immediately after the first which returned the
data. As no more data will be received, this other read just blocks
forever. fgets happens to work because the call returns after a complete
line has been read. But AFAIK, that's not mandated behaviour and a
different implementation may well hang in fgets, too.

The basic problem is that you're doing real-time I/O using an
I/O-facility not intended to operate in real-time. If real-time
operation is desired, which is generally the case for all kinds of
request-response protocol operations, you'll need to use read/write or
send/recv.

[Lew Pitcher]

Post by Rainer Weikusat

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.

I did some tests with this with the built-in inetd echo server and the
fscanf, regardless of buffering mode the stream has been set to, simply
does another read immediately after the first which returned the
data. As no more data will be received, this other read just blocks
forever. fgets happens to work because the call returns after a complete
line has been read. But AFAIK, that's not mandated behaviour and a
different implementation may well hang in fgets, too.
The basic problem is that you're doing real-time I/O using an
I/O-facility not intended to operate in real-time. If real-time
operation is desired, which is generally the case for all kinds of
request-response protocol operations, you'll need to use read/write or
send/recv.

Thanks, Rainer

As I suspected, stdio on sockets is tricky, and not the correct interface
to use. But, I had to try :-)

Thanks again for your help

[Ben Bacarisse]

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.
BUT, if I
char buffer[1024], vrm[16];
fgets(buffer,sizeof buffer,fd);
sscanf(buffer,"Asterisk Call Manager/%s\r\n",vrm);
I manage to obtain the appropriate data. A dump of the
buffer array shows that I did indeed capture the introduction line
from the server.
So, why the difference in behaviour?

You've had some good advice, but no one has answered the question.

The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character. That's more reading than you want in a
protocol stream.

Though you might be able to fix it (you could read exactly two single
characters and check that they are \r and \n) it's generally better to
read a response and use sscanf if that suits the application

(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

Post by Lew Pitcher
Obviously, in the fscanf() version,
I've not set something up right. Any hints as to what I've done wrong
would be greatly appreciated.

[Lew Pitcher]

Post by Ben Bacarisse

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.

[snip]

Post by Ben Bacarisse

Post by Lew Pitcher
So, why the difference in behaviour?

You've had some good advice, but no one has answered the question.
The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character. That's more reading than you want in a
protocol stream.

AHA! That makes sense. Thanks, Ben, for the clarification. I had forgotten
that fscanf() treats \r and \n as whitespace. And, that explains why the
sscanf() doen't fail; it encounters the end-of-string and treats it as
the equivalent of an EOF or non-space character. And so, doesn't hang.

Post by Ben Bacarisse
Though you might be able to fix it (you could read exactly two single
characters and check that they are \r and \n) it's generally better to
read a response and use sscanf if that suits the application

Again, that makes sense. A good suggestion.

Post by Ben Bacarisse
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot. So,
for now, I'll stick with blocking I/O and a foreknowledge of what
is (or should be) coming down the wire. To me, at this time,
non-blocking I/O is like adding a regex to the mix ("And, now you
have /two/ problems." :-) )

Thanks for the advice. It certainly explains the differences between
the three trial programs, and gives me something to think about wrt
how I handle this sockets I/O.

[Lew Pitcher]

Post by Lew Pitcher

Post by Ben Bacarisse

Post by Lew Pitcher
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.

[snip]

Post by Ben Bacarisse

Post by Lew Pitcher
So, why the difference in behaviour?

You've had some good advice, but no one has answered the question.
The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character. That's more reading than you want in a
protocol stream.

AHA! That makes sense. Thanks, Ben, for the clarification. I had forgotten
that fscanf() treats \r and \n as whitespace. And, that explains why the
sscanf() doen't fail; it encounters the end-of-string and treats it as
the equivalent of an EOF or non-space character. And so, doesn't hang.

And BINGO, when I change the line from
fscanf(amiIn,"Asterisk Call Manager/%s\r\n",vrm)
to
fscanf(amiIn,"Asterisk Call Manager/%s",vrm)
I get the expected results, and confirm Ben's observation.

Thanks again, Ben
That solved my puzzle.

And, now on to reworking my design, based on the advice I've seen here.

Thanks, all, for the help.

[Rainer Weikusat]

[...]

Post by Lew Pitcher

Post by Ben Bacarisse
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot.

The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time input¹ on more than one file descriptor and thus, cannot block
waiting for input on one of them as data might arrive at one of the
others first. Hence, it'll set all to non-blocking and then blocks
select/ poll/ epoll etc so that it gets notified when input is available
on any of them.

A somewhat simpler use case is I/O on only one file descriptor but with
a need to impose timeouts to cope with unreliable communication
partners, although there are other options available for this (like
alarm signals or socket read timeouts).

¹ Principally also to avoid blocking for buffer space to become
available to complete an output operations but that's a rare case.

[Lew Pitcher]

Post by Rainer Weikusat
[...]

Post by Lew Pitcher

Post by Ben Bacarisse
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot.

The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time input¹ on more than one file descriptor and thus, cannot block
waiting for input on one of them as data might arrive at one of the
others first.

[snip]

Post by Rainer Weikusat
A somewhat simpler use case is I/O on only one file descriptor but with
a need to impose timeouts to cope with unreliable communication
partners

[snip]

For this mini-project, both client and server live on the same machine,
and communicate through the loopback ("localhost"). There won't be
any significant latency issues or other communications interference.

I'm only concerned with writing the client, and it will only interact
over one channel in a simple request/reply type protocol. There doesn't
seem to be a pressing need for nonblocking I/O.

But, as it has been suggested here, I will look into nonblocking I/O.
Perhaps I'm missing something that others have seen.

[Rainer Weikusat]

Post by Lew Pitcher

Post by Rainer Weikusat
[...]

Post by Lew Pitcher

Post by Ben Bacarisse
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot.

The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time input¹ on more than one file descriptor and thus, cannot block
waiting for input on one of them as data might arrive at one of the
others first.

[snip]

Post by Rainer Weikusat
A somewhat simpler use case is I/O on only one file descriptor but with
a need to impose timeouts to cope with unreliable communication
partners

[snip]
For this mini-project, both client and server live on the same machine,
and communicate through the loopback ("localhost"). There won't be
any significant latency issues or other communications interference.
I'm only concerned with writing the client, and it will only interact
over one channel in a simple request/reply type protocol. There doesn't
seem to be a pressing need for nonblocking I/O.
But, as it has been suggested here, I will look into nonblocking I/O.
Perhaps I'm missing something that others have seen.

If you don't want to implement I/O timeouts, non-blocking I/O is uselees
for your case. I think it's often just being (mis-)used because it exists and
the idea of a program which just waits until woken up by the kernel
instead of Doing Someting[tm] make many people nervous.

[Tim Rentsch]

Post by Lew Pitcher
[..]

[...]

Post by Lew Pitcher

Post by Lew Pitcher

Post by Ben Bacarisse
(The only advice I'd question is that of using non-block I/O by
default. I would be surprised if that helps you in this project,
and it might make things more complicated.)

I've done less programming with non-blocking I/O than I've done
with sockets. I have heard the advice to use non-blocking I/O,
and will consider it, but I don't want to take on too much in one
shot.

[...]

For this mini-project, both client and server live on the same
machine, and communicate through the loopback ("localhost").
There won't be any significant latency issues or other
communications interference.
I'm only concerned with writing the client, and it will only
interact over one channel in a simple request/reply type protocol.
There doesn't seem to be a pressing need for nonblocking I/O.
But, as it has been suggested here, I will look into nonblocking
I/O. Perhaps I'm missing something that others have seen.

I guess I should expand on the idea that using non-blocking I/O
is a good idea.

First, my comment was offered as pragmatic advice following many
years of experience both with networking and with socket-level
programming. It is generic advice, given without taking into
account any of the specifics of your project. There is no question
that using non-blocking I/O means more work up front, and I expect
there is a good chance you could get something working without doing
any of the non-blocking stuff.

That said, here are some things to think about.

Although there is only one channel, there are two communication
paths: one for reading and one for writing. If there were really
only one path (e.g., reading a file and writing to a socket) then
using blocking I/O is probably good enough. But any more than
just a single path makes things more complicated.

There can be problems at the protocol level even if the transport
layer is working perfectly. Network programming has a lot in
common with inter-process synchronization, and is just as tricky
except in the cases where it's even trickier. How is the remote
process (which is Asterisk in this case) going to behave? That
can (and often does) matter.

(Incidentally, speaking of Asterisk, Asterisk is great! I've
been running my own home phone system on Asterisk for more than a
decade now, and I'd never give it up.)

As far as the transport layer goes, TCP is very reliable once it
gets going, but there is a little dance that happens at the start
to get things set up, and sometimes one of the partners steps on
the other's foot. I see this happen sporadically with programs
that have the same basic outline as your application (one socket
used for both reading and writing). Probably there is some sort
of interaction with what's going on at the protocol level, but
it's hard to know whether the problem is a protocol issue or
something in one of the network layers.

Assuming you do go ahead with using blocking I/O, and get things
working, there is still a fair chance that at some point down the
road (probably after the application has been deployed) the program
will be the victim of a network hiccup and go catatonic. If and
when that happens: one, it will be frustrating; two, the problem
will be non-reproducible; and three, you won't have any tools to
help diagnose what's going on to cause the problem, because all the
significant events are happening inside one of the blocking system
calls. Conversely, if all the socket I/O is non-blocking, the code
can be instrumented (whether before the fact or after) with various
sorts of logging, and the log information can be examined to see
what's going on.

Final point: probably the most important lesson here is that we
may expect that network system calls are going to fail, and should
program accordingly. When working with files it's easy to get
into the habit of thinking the calls will never fail, because they
almost never do. Network system calls are different. They might
not fail all the time, but they *are* going to fail, and it's
better to anticipate that, and program accordingly. Using
non-blocking I/O makes that easier, partly because "failures"
happen more often in the form of "operation would block", so one
gets into the habit of writing code that handles different kinds
of error returns. This means a little more work at the outset,
but ultimately less work (we hope!) overall.

I guess I should add that I'm not trying to persuade anyone, and
please go ahead as you think best. I thought it might be helpful
to explain some of the rationale underlying my suggestion, and
hence this response.

[Rainer Weikusat]

Tim Rentsch <***@z991.linuxsc.com> writes:

[...]

Post by Tim Rentsch
I guess I should expand on the idea that using non-blocking I/O
is a good idea.
First, my comment was offered as pragmatic advice following many
years of experience both with networking and with socket-level
programming. It is generic advice, given without taking into
account any of the specifics of your project.

[...]

Ie, this is about using a certain model for handling I/O (specifically,
as originally suggested, busy-polling with interspersed sleeps)
regardless of the actual needs of an application.

[long general statement whose relevance I don't understand]

Post by Tim Rentsch
Assuming you do go ahead with using blocking I/O, and get things
working, there is still a fair chance that at some point down the
road (probably after the application has been deployed) the program
will be the victim of a network hiccup and go catatonic.

That's pretty much guaranteed to happen as all kinds of 'network
demolition devices' (commonly called firewalls) will silently cut off
perfectly working TCP connections, either because these have exceeded
some idle time their masters considered just too much or because they
have been rebooted or are malfunctioning. But this is only relevant for
long-running, non-interactive programs and will usually need some scheme
for periodic keepalive exchanges to detect connections which have
silenty gone bad.

Post by Tim Rentsch
If and when that happens: one, it will be frustrating; two, the problem
will be non-reproducible; and three, you won't have any tools to
help diagnose what's going on to cause the problem, because all the
significant events are happening inside one of the blocking system
calls. Conversely, if all the socket I/O is non-blocking, the code
can be instrumented (whether before the fact or after) with various
sorts of logging, and the log information can be examined to see
what's going on.

With non-blocking I/O, all significant events still happen inside the
kernel, it's just that the program is hammering the kernel with
avoidable system calls.

Post by Tim Rentsch
Final point: probably the most important lesson here is that we
may expect that network system calls are going to fail, and should
program accordingly. When working with files it's easy to get
into the habit of thinking the calls will never fail, because they
almost never do. Network system calls are different. They might
not fail all the time, but they *are* going to fail, and it's
better to anticipate that, and program accordingly. Using
non-blocking I/O makes that easier, partly because "failures"
happen more often in the form of "operation would block", so one
gets into the habit of writing code that handles different kinds
of error returns. This means a little more work at the outset,
but ultimately less work (we hope!) overall.

It's perfectly possible to implement handling for network errors without
introducing EAGAIN pseudo-errors in situations where they don't make
much sense (such as using a request-response protocol over a single TCP
connection).

[Nicolas George]

Rainer Weikusat , dans le message

Post by Rainer Weikusat
The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time input¹ on more than one file descriptor and thus, cannot block
waiting for input on one of them as data might arrive at one of the
others first.

While true, your statement is misleading by bringing threads into the
discussion: it seems to imply that threads would be a solution to use the
convenience of blocking I/O with real-time programs.

But the idea that threads solve I/O concurrency issues is a misconception.
The design POSIX thread API entirely ignores the file descriptor API, and
therefore instead of helping making fd I/O easier, it makes it harder.
People who try to solve concurrent I/O with threads achieve something first,
but as soon as they start implementing the necessary features like timeout
they realize they need to put an event-loop, poll()-based or other, in most
of their threads, while they hoped to avoid an event-loop in their
single-threaded program.

POSIX threads are useful for performance, but it is a mistake to think they
help for I/O. And that mistake is pervasive enough that we must be careful
not to amplify it.

[Lawrence D'Oliveiro]

Post by Nicolas George
POSIX threads are useful for performance, but it is a mistake to think
they help for I/O. And that mistake is pervasive enough that we must be
careful not to amplify it.

The 1990s were very much the “decade of threads”, weren’t they. The
concept was just starting to become pervasive across the OSes of the time,
and people wanted to use them for everything, even GUIs.

Multithreading in GUIs (e.g. Sun’s NeWS) turned out to be a mistake.

[Rainer Weikusat]

[...]

Post by Ben Bacarisse

Post by Lew Pitcher
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.

[...]

Post by Ben Bacarisse
The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character.

This is specifically demanded by the UNIX specification:

,----
| A directive composed of one or more white-space characters shall be
| executed by reading input until no more valid input can be read, or up
| to the first byte which is not a white-space character, which remains
| unread.
`----

OTOH, the stream may be fully buffered or line buffered (unspecified)
and - AFAICT - no specific behaviour is manadated for input from either
a fully buffered or a line buffered stream. Eg, the implementation is
free to do readahead to whatever degree someone considers useful. Even
for an unbuffered stream, it's just said that

,----
| bytes are intended to appear from the source or at the destination as
| soon as possible;
`----

which means setting a stream to unbuffered is basically a hint to the
implementation: If that's deemed possible, please send the data now or, for
input, return as soon as enough the call can complete successfully and
the implementation is totally free to take a Kinderwille ist Dreck!¹
stance on that.

¹ German for "Children may wish for anything. Doesn't mean they'll get
it!"

[Keith Thompson]

Post by Rainer Weikusat
[...]

Post by Ben Bacarisse

Post by Lew Pitcher
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.

[...]

Post by Ben Bacarisse
The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character.

,----
| A directive composed of one or more white-space characters shall be
| executed by reading input until no more valid input can be read, or up
| to the first byte which is not a white-space character, which remains
| unread.
`----

And by the ISO C standard:

A directive composed of white-space character(s) is executed by
reading input up to the first non-white-space character (which
remains unread), or until no more characters can be read. The
directive never fails.
[...]

[Lawrence D'Oliveiro]

Post by Ben Bacarisse
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

This is why we have the “async” paradigm, a.k.a. the resurgence of
coroutines. It leads to fewer race conditions than threads, and works well
where the bottleneck is network I/O (or, in a GUI app, the user
interaction), rather than the CPU.

[Rainer Weikusat]

Post by Lawrence D'Oliveiro

Post by Ben Bacarisse
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

This is why we have the “async” paradigm, a.k.a. the resurgence of
coroutines.

From coroutines, it's usually one step until somebody again reimplements
cooperative userspace threading. Because of this, I'll take the liberty
here to give a brief (hopefully) abstract description of something I've
found useful for modelling complex asynchronous I/O operations¹. This is
in Perl but should be applicable to any language which supports
closures. It's structured around poll (entirely sufficient for small
numbers of file descriptors) but could be made to work with 'something
else' (eg, epoll) as well.

At the base of this is an I/O completion object I've called a 'want' (as
it describes something a program wants). This object contains a file
handle/ file descriptor, an event the program wants (input available or
output possible) and a continuation closure which should be invoked
after the even has occurred.

At the beginning of each iteration of the main event handling loop,
there's a set of current wants and a poll call is made waiting for any
of the I/O events indicated by these. After something happened and this
call returned, the code goes through the set of active wants and
construct a set of next wants as follows:

If the event a certain want wanted hasn't happened yet, add it to the
next set. If it has happened, call the continuation closure which
returns a possibly empty set of wants. If the set isn't empty, add all
of these to the set of next wants. After all active wants have been
dealt with in this way, make the next set the active set and start the
next iteration of the loop.

¹ Eg, to a REST request. This requires a DNS lookup, establishing a TCP
connection, doing a TLS handshake, sending the request and processing
the reply. This involves 2 different file descriptors and multiple I/O
event 'mode switches', waiting for a different kind of event for a file
descriptor than the last one. Establishing a TCP connection
asynchronously is a 'ready for write' event. The following TLS handshake
will at least need to wait for 'input available' but may need both of
them in some a priori unpredictable sequence.

[Lawrence D'Oliveiro]

Post by Rainer Weikusat

Post by Lawrence D'Oliveiro
This is why we have the “async” paradigm, a.k.a. the resurgence of
coroutines.

From coroutines, it's usually one step until somebody again reimplements
cooperative userspace threading.

That’s what “async” is all about. In Python asyncio, they use the term
“tasks” for these cooperative schedulable entities. They are wrappers
around Python coroutine objects.

Post by Rainer Weikusat
At the base of this is an I/O completion object I've called a 'want' (as
it describes something a program wants). This object contains a file
handle/ file descriptor, an event the program wants (input available or
output possible) and a continuation closure which should be invoked
after the even has occurred.

In the Python asyncio library, there is this concept of a “future”. In
JavaScript, they call it a “promise”. This is not tied in any way to
particular open files or anything else: think of it as a box, initially
empty, with a sign above it saying “watch this space”. At some point,
something should happen, and tasks can block on waiting for it. When
something appears in the box, they are woken up and can retrieve a copy of
the contents.

In Python asyncio, you can also set an exception on the future. So instead
of getting back a value, each waiting task gets the exception raised in
their context.

Post by Rainer Weikusat
At the beginning of each iteration of the main event handling loop,
there's a set of current wants and a poll call is made waiting for any
of the I/O events indicated by these.

Not just I/O events, you also want timers as well.

[Rainer Weikusat]

Post by Lawrence D'Oliveiro

Post by Rainer Weikusat

Post by Lawrence D'Oliveiro
This is why we have the “async” paradigm, a.k.a. the resurgence of
coroutines.

From coroutines, it's usually one step until somebody again reimplements
cooperative userspace threading.

That’s what “async” is all about. In Python asyncio, they use the term
“tasks” for these cooperative schedulable entities. They are wrappers
around Python coroutine objects.

No, it's not. That's an overgeneralisation of a useful concept people
keep reinventing (I just found another one or two weeks ago).

Post by Lawrence D'Oliveiro

Post by Rainer Weikusat
At the base of this is an I/O completion object I've called a 'want' (as
it describes something a program wants). This object contains a file
handle/ file descriptor, an event the program wants (input available or
output possible) and a continuation closure which should be invoked
after the even has occurred.

In the Python asyncio library, there is this concept of a “future”. In
JavaScript, they call it a “promise”. This is not tied in any way to
particular open files or anything else: think of it as a box, initially
empty, with a sign above it saying “watch this space”. At some point,
something should happen, and tasks can block on waiting for it. When
something appears in the box, they are woken up and can retrieve a copy of
the contents.

And that's another overgeneralization, see below.

[...]

Post by Lawrence D'Oliveiro

Post by Rainer Weikusat
At the beginning of each iteration of the main event handling loop,
there's a set of current wants and a poll call is made waiting for any
of the I/O events indicated by these.

Not just I/O events, you also want timers as well.

On Linux, timers and - more generally - other signal-driven things are
I/O events. It's possible to receive signals on file descriptor via
signalfd which - in combination with setitimer, can be used to provide
arbitrary timer events. It's also possible to get timer events directly
via file descriptor by using 'timer' fds (=> timerfd_create and
friends). I haven't found a real use for these so far,
though. Programming alarms and using these to activate timers based on
some data structure for storing them (eg, a sorted linked list or a
more elaborate priority queue) has been sufficient for me so far.

[Lawrence D'Oliveiro]

Post by Rainer Weikusat
That's an overgeneralisation of a useful concept people
keep reinventing ...

If it really was a generalization, then there would be nothing to
“reinvent”. You only need to “reinvent” something if a prior concept was
not general enough.

TL;DR: “overgeneralization ... you keep using that word ... I do not think
it means what you think it means”.

[Rainer Weikusat]

Post by Lawrence D'Oliveiro

Post by Rainer Weikusat
That's an overgeneralisation of a useful concept people
keep reinventing ...

If it really was a generalization, then there would be nothing to
“reinvent”. You only need to “reinvent” something if a prior concept was
not general enough.

Surely, as someone who claims to have programmed something the past, you
must be aware that it's possible to reimplement or reprogram something
somebody else already implemented/ programmed in the past, ie, creating (yet
another) green threads implementation.

Post by Lawrence D'Oliveiro
TL;DR: “overgeneralization ... you keep using that word ... I do not think
it means what you think it means”.

If you think that you're thinking, you're only thinking that you're
thinking.

Yes, this is silly. But so is becoming abusive within two posts for
absolutely no reason.

[Kaz Kylheku]

Post by Lawrence D'Oliveiro

Post by Rainer Weikusat
That's an overgeneralisation of a useful concept people
keep reinventing ...

If it really was a generalization, then there would be nothing to
“reinvent”. You only need to “reinvent” something if a prior concept was
not general enough.

Nope! How it works in computing is that people "reinvent" something if
they have no clue it existed before, and describe it using different
terminology.