Discussion:
File (terminal) is in /dev/pts, but nobody has it open???
(too old to reply)
Kenny McCormack
2021-04-19 17:51:15 UTC
Permalink
This is on a home Linux system, so there is only one user (me) and I have
root, so there is no issue with access.

I've noticed, on one particular machine, that sometimes there will be a
file, say /dev/pts/6, in /dev/pts, but there is no process running on that
device and nobody has it open. My understanding is that the file should be
purged when the last process that had it open closes it (or exits).

I verify that nobody has it open by doing: sudo fuser /dev/pts/6
and it reports nothing. My general guess as to the problem is that the
process that had it open exited, but whatever background process is
responsible for cleaning up /dev/pts didn't notice. Note also that I
cannot remove the file manually (even as root!)

The effect of this is that that pts is basically "lost", until the next
reboot. I want to know a) How/why this happens and b) If there is anything
I can do to get it back.

Note: This may sound like another, similar, problem, but it is not.
Specifically, I want to assure you that I am quite familiar with what you
might call the opposite problem to this one. That is, when there is no
file in /dev/pts for a given terminal, yet, when you allocate a new one,
you get some other number. This is caused by some process having the file
open, but somehow it gets deleted from /dev/pts even though somebody still
has it open. You can detect (and fix) this problem by doing:

# sudo ls -lsa /proc/*/fd | less

and searching for the word "deleted".

Note that, in this case, you can't use fuser to find the offending process,
because there is no file in /dev/pts to use as the argument to fuser.
--
Trump has normalized hate.

The media has normalized Trump.
Scott Lurndal
2021-04-19 17:58:17 UTC
Permalink
Post by Kenny McCormack
This is on a home Linux system, so there is only one user (me) and I have
root, so there is no issue with access.
I've noticed, on one particular machine, that sometimes there will be a
file, say /dev/pts/6, in /dev/pts, but there is no process running on that
device and nobody has it open. My understanding is that the file should be
purged when the last process that had it open closes it (or exits).
It's not a file, per-se.

/dev/pts is managed by the 'ptsfs' filesystem.

$ mount | grep pts
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)

There is likely a reference to the pty somewhere; try using the 'w' command.

We use thousands of pty's every day with no leakage issues.
Kaz Kylheku
2021-04-19 18:54:14 UTC
Permalink
Post by Scott Lurndal
Post by Kenny McCormack
This is on a home Linux system, so there is only one user (me) and I have
root, so there is no issue with access.
I've noticed, on one particular machine, that sometimes there will be a
file, say /dev/pts/6, in /dev/pts, but there is no process running on that
device and nobody has it open. My understanding is that the file should be
purged when the last process that had it open closes it (or exits).
It's not a file, per-se.
/dev/pts is managed by the 'ptsfs' filesystem.
$ mount | grep pts
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
There is likely a reference to the pty somewhere; try using the 'w' command.
We use thousands of pty's every day with no leakage issues.
Linux is so awfully fragmented that you cannot say anything for sure,
though.

Most people use a vendor specific kernel with patches, and no two of
those are alike even if the upstream version numbers match.

Maybe someone tampered in the pty code in Kenny's kernel so there is a
genuine leak.
--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Scott Lurndal
2021-04-19 18:00:26 UTC
Permalink
Post by Kenny McCormack
This is on a home Linux system, so there is only one user (me) and I have
root, so there is no issue with access.
More specifically, the "dangling" pts entry is like because
it remains a controlling TTY for some process somewhere. fuser
may or may not find it, but

$ sudo lsof | grep pts

should find it for you.
Richard Kettlewell
2021-04-19 18:33:34 UTC
Permalink
Post by Kenny McCormack
This is on a home Linux system, so there is only one user (me) and I have
root, so there is no issue with access.
I've noticed, on one particular machine, that sometimes there will be a
file, say /dev/pts/6, in /dev/pts, but there is no process running on that
device and nobody has it open. My understanding is that the file should be
purged when the last process that had it open closes it (or exits).
I verify that nobody has it open by doing: sudo fuser /dev/pts/6
and it reports nothing. My general guess as to the problem is that the
process that had it open exited, but whatever background process is
responsible for cleaning up /dev/pts didn't notice. Note also that I
cannot remove the file manually (even as root!)
The effect of this is that that pts is basically "lost", until the next
reboot. I want to know a) How/why this happens and b) If there is anything
I can do to get it back.
The /dev/pts file is removed when all file descriptors on either end of
the pty are closed.

For example, if the pty belongs to an SSH session, then both the command
running inside the SSH session, _and_ the sshd process for that session,
must terminate before the pty is closed.

The sshd’s end of the pty will normally appear as /dev/ptmx rather than
/dev/pts/N, reflecting the pty creation process. I’ve reproduced the
behavior you see by sending SIGSTOP to an sshd and then terminating its
child.

To get the pty back, find the process holding the relevant copy of
/dev/ptmx open and (if it’s not serving some useful purpose) kill it.

There’s insufficient information to speculate why it happens in your
particular environment.
--
https://www.greenend.org.uk/rjk/
Kenny McCormack
2021-04-19 19:41:27 UTC
Permalink
In article <***@LkoBDZeT.terraraq.uk>,
Richard Kettlewell <***@invalid.invalid> wrote:
...
Post by Richard Kettlewell
The /dev/pts file is removed when all file descriptors on either end of
the pty are closed.
Yes, that is what is SUPPOSED TO HAPPEN.

But is isn't (always). fuser and lsof both say noone has it open.

I know this is hard for people responding in the newsgroup to accept, but
it doesn't always work the way it is supposed to.

Most of what has been posted so far in the resposes to this thread is stuff
I (obviously) already know.
--
Debating creationists on the topic of evolution is rather like trying to
play chess with a pigeon --- it knocks the pieces over, craps on the
board, and flies back to its flock to claim victory.
Richard Kettlewell
2021-04-19 20:52:53 UTC
Permalink
Post by Kenny McCormack
...
Post by Richard Kettlewell
The /dev/pts file is removed when all file descriptors on either end of
the pty are closed.
Yes, that is what is SUPPOSED TO HAPPEN.
But is isn't (always). fuser and lsof both say noone has it open.
You said you did ‘sudo fuser /dev/pts/6’, which only checks one end. If
you knew about checking for the other end as well then perhaps you
should have mentioned that, rather than expecting your readers to be
telepathic.
--
https://www.greenend.org.uk/rjk/
Kenny McCormack
2021-04-20 05:22:53 UTC
Permalink
Post by Richard Kettlewell
Post by Kenny McCormack
...
Post by Richard Kettlewell
The /dev/pts file is removed when all file descriptors on either end of
the pty are closed.
But is isn't (always). fuser and lsof both say noone has it open.
checking for the other end as well
OK, I tried a few things, but could not figure out how to check for that.
Any ideas?

Lots of processes have /dev/ptmx open; how to know which one is connected
to /dev/pts/6?
--
"Every time Mitt opens his mouth, a swing state gets its wings."

(Should be on a bumper sticker)
Richard Kettlewell
2021-04-20 08:34:33 UTC
Permalink
Post by Kenny McCormack
Post by Richard Kettlewell
Post by Kenny McCormack
...
Post by Richard Kettlewell
The /dev/pts file is removed when all file descriptors on either end of
the pty are closed.
But is isn't (always). fuser and lsof both say noone has it open.
checking for the other end as well
OK, I tried a few things, but could not figure out how to check for that.
Any ideas?
Lots of processes have /dev/ptmx open; how to know which one is connected
to /dev/pts/6?
I don’t know of a way to discover it directly; you will have to look
through all the processes with ptmx and figure out which is the relevant
one from contextual clues. e.g. sshd’s argv, or if there are no child
processes (or only child processes with no tty) that’s a good
candidate. I’d have thought someone who knows how it’s all supposed to
work could figure out this approach quite easily for themselves though.
--
https://www.greenend.org.uk/rjk/
Zooe
2021-04-20 11:26:17 UTC
Permalink
Post by Kenny McCormack
Lots of processes have /dev/ptmx open; how to know which one is connected
to /dev/pts/6?
For each fd in /proc/$pidmaster/fd/ that points to /dev/ptmx, check the
following (works only on recent kernels):

grep tty-index < /proc/$pidmaster/fdinfo/$fd

Another way to get the information, which is more universal but also is
more difficult to follow, is to make a TIOCGPTN system call on the file
descriptor (from a debugger attached to the process).
Kenny McCormack
2021-04-20 12:42:15 UTC
Permalink
Post by Zooe
Post by Kenny McCormack
Lots of processes have /dev/ptmx open; how to know which one is connected
to /dev/pts/6?
For each fd in /proc/$pidmaster/fd/ that points to /dev/ptmx, check the
grep tty-index < /proc/$pidmaster/fdinfo/$fd
Ah, now we are getting somewhere.

Unfortunately, the system that has the problem is running kernel
4.something and does not have tty-index (in the fdinfo/X file). Other
systems here that are runnning kernel 5.something do have it, but "man
proc" doesn't mention it.

I'm guessing that it contains the number of the tty (/dev/pts/X) that this
instance of /dev/ptmx is connected to. Is that correct?
Post by Zooe
Another way to get the information, which is more universal but also is
more difficult to follow, is to make a TIOCGPTN system call on the file
descriptor (from a debugger attached to the process).
Well, that sounds interesting. And very complicated...
(And time consuming...)
--
The coronavirus is the first thing, in his 74 pathetic years of existence,
that the orange menace has come into contact with, that he couldn't browbeat,
bully, bullshit, bribe, sue, legally harrass, get Daddy to fix, get his
siblings to bail him out of, or, if all else fails, simply wish it away.
Zooe
2021-04-20 17:32:49 UTC
Permalink
Post by Kenny McCormack
Post by Zooe
grep tty-index < /proc/$pidmaster/fdinfo/$fd
Ah, now we are getting somewhere.
Unfortunately, the system that has the problem is running kernel
4.something and does not have tty-index (in the fdinfo/X file). Other
systems here that are runnning kernel 5.something do have it, but "man
proc" doesn't mention it.
I'm guessing that it contains the number of the tty (/dev/pts/X) that this
instance of /dev/ptmx is connected to. Is that correct?
Answer is yes.

The patch that implements the feature was applied on July 2017.
It is documented here: https://lore.kernel.org/patchwork/patch/810490/
Post by Kenny McCormack
Post by Zooe
Another way to get the information, which is more universal but also is
more difficult to follow, is to make a TIOCGPTN system call on the file
descriptor (from a debugger attached to the process).
Well, that sounds interesting. And very complicated...
(And time consuming...)
I once saw a perl script automating this, but I cannot find it anywhere.
Rainer Weikusat
2021-04-20 22:12:08 UTC
Permalink
***@shell.xmission.com (Kenny McCormack) writes:

[...]
Post by Kenny McCormack
Post by Zooe
Another way to get the information, which is more universal but also is
more difficult to follow, is to make a TIOCGPTN system call on the file
descriptor (from a debugger attached to the process).
Well, that sounds interesting. And very complicated...
(And time consuming...)
It's really quite simple. First, the numeric value of TIOCGPTN needs to
be known. The following program will print it:

------
#include <stdio.h>
#include <sys/ioctl.h>

int main(int argc, char **argv)
{
printf("%u\n", TIOCGPTN);
return 0;
}
------

On my system, it's 2147767344. Then, one needs to start gdb with the
program file as argument, eg

gdb /sbin/sshd

and use the attach command to attach it to the process one wants to
examinate:

(gdb) attach 9206

This will print a lengthy blurb. The ioctl needs an int * as argument,
hence, one needs to call malloc to get one ($p is a gdb "convenience
variable")

(gdb) set $p = malloc(4)

check that it succeeded

(gdb) p $p
$1 = (void *) 0x7f9f2a50f8e0

run the ioctl with the proper file descriptor number

(gdb) p (int)ioctl(10, 2147767344, $p)
$2 = 0

print the value

(gdb) p *(int *)$p
$3 = 3

finally, free the memory again:

(gdb) p (void)free($p)
$4 = void

and exit the debugger.

The sought-after pty number is thus 3.
Kenny McCormack
2021-04-20 22:22:53 UTC
Permalink
[...]
Post by Kenny McCormack
Post by Zooe
Another way to get the information, which is more universal but also is
more difficult to follow, is to make a TIOCGPTN system call on the file
descriptor (from a debugger attached to the process).
Well, that sounds interesting. And very complicated...
(And time consuming...)
It's really quite simple. First, the numeric value of TIOCGPTN needs to ...
Thanks for posting that. I have done that sort of thing in the past, but
couldn't quite remember the jigs and jogs of it. FWIW, I find both gdb and
its documentation incomprehensible.

Anyway, it looks like you could also do it by using ptsname(3). I was
thinking of doing it along those lines - instead of using the ioctl().

Also, once one has the method down, I would write an Expect script to
automate the process (iterate through all the fds of all the processes that
have /dev/ptms open).
--
Joni Ernst (2014): Obama should be impeached because 2 people have died of Ebola.
Joni Ernst (2020): Trump is doing great things, because only 65,000 times as many people have died of COVID-19.

Josef Stalin (1947): When one person dies, it is a tragedy; when a million die, it is merely statistics.
Loading...