Little Black Lights

mommy, look, over there!

ElCondor

[aka Martin Schipany]

Preface

• you're in the wrong talk if ...
• you are using a non-un*x OS
• you know how to make signals unsafe or restartable
• it's Un*x based only (POSIX, BSD, System-V)
• The Players:
  • Kernel,
  • Libraries,
  • Perl itself,
  • your code
• Pseudo-Code included(to keep it short)

Black Lights?

'When you try and operate one of these weird black controls which are labeled in black on a black background, a small black light lights up black to tell you that you've done it'

Zaphod Beeblebrox

Signals are always there, telling you that or even what happened, and that it might as well be just a good idea to take some action on this. But it's up to you to do so. And you are free to fail with style

What are signals (good for)?

• Signals are means of communication are supplied by the operating system (real ones)
• IPC: InterProcessCommunication:
• Signals are a method for one process to notify another about something that happened.
• any process will receive signals
• any process can send signals
• each signal has a name and a number (ANSI, POSIX, BSD, System-V)
• 'kill' command sends signals (shell and perl)
• beware: signals are easy to use, but TRICKY to control!

Types of Signals

examples most commonly used signals

• Interrupts 1 SIGHUP 2 SIGINT 15 SIGTERM

  Controls 10 SIGUSR1 12 SIGUSR2 13 SIGPIPE 17 SIGCHLD

  special 9 SIGKILL SIGSTOP 14 SIGALRM

• full list of available system signals:
  'kill -l' (or perl -le 'print join(" ", keys %SIG)')
• Perl internal signals
• __DIE__
• __WARN__

How to send/Issue Signals

• kill command sends signals (shell and perl)
• Shell:
  • kill -1 12345
  • kill -HUP 12345
  • (killall -KILL myprog)
• Perl
  • kill 1 => $pid
  • kill USR1 => @children
  • kill -15 => $job_by_group
• by external cause
• programm on end of pipe is gone, nfs disappeared,
• exceeding of cpu time or file size limit
• division by zero, memory access violation

How To Process Signals

%SIG: code in that hash will be automatically processed in case the corresponding signal is triggered

• $SIG{PIPE} = 'handle_sig_pipe' # means main::handle_sig_pipe is called
• $SIG{INT} = \&handle_int;
• $SIG{USR1} = sub { say 'that tickles!'; }
• $SIG{HUP} = 'IGNORE';
• $SIG{TERM} = 'DEFAULT';

How to use that?

Let's take a look at some examples:

• Enemy of the commandline: Ctrl + C
• Whom the Bell tolls: Alarm!
• Who wants to live forever? DIE!
• Ghostbuster 1: Commanding a Daemon
• Ghostbuster 2: Spawn and Reap
• Caveats
• More Light

Enemy of the commandline

We just want to take CTRL-C it's power, nothing fancy

• sub important {
      local $SIG{INT} = \&sig_int;
      # do some horrible important job here!
      # that should not be interrupted
  }

• sub sig_int {
      local $SIG{INT} = \&sig_int; # o que é isto?
      print "hey, don't interrupt me! .. \n";
  }

Whom the Bell tolls

We have a task that might take forever, but we want to limit the time it may take.

• eval {
      local $SIG{ALRM}= sub { die "took ages"; }
      alarm $timeout; # int, seconds
      $error = &this_make_take_ages(); # LWP?
      $timeleft= alarm 0;
      1;
  } or do {
      if ($@ =~ /took ages/) {
          # take care of data ...
      }
      else { # something else went wrong ..
          Carp::longmess(@_);
      }};

• mind: only one alarm per process

Who wants to live forever

Time and again, every good program is going to die. Gathering more information should be the goal. or even more ...

• $SIG{'__DIE__'} = sub {
        log_sys->fatal( "PERL DIE: " .
              Carp::longmess(@_) );
        die @_; # bail out ..
      };

• always use some exeption handling
• spend enough time on good exeption handling!
• Resurrection: returning after __DIE__ ?

Controlling a Daemon

We start a program to be run in the background. Let's assume is has some important job to to, that may not be interrupted without saving the current state. Furthermore we want it to write out some data when we tell it to.

• $SIG{TERM}= \&sig_term;
  $SIG{HUP}= \&sig_hup;

• sub sig_term {
      $SIG{TERM}= 'IGNORE';
      &collect_data_and_write_buffers;
      exit 1; }

• sub sig_hup {
      $SIG{HUP}= 'IGNORE';
      &write_stats_to_usbdrive;
      $SIG{HUP}= \&sig_hup; }

• what if ... HUP then TERM ?

Spawn ..

We need a bunch of processes, so let's fork some children:

$SIG{CHLD} = \&reaper;
if ( $pid = fork() ) {
    push @children,$pid; # remember
    return $pid;
}
elsif ( defined $pid ) {
    &write_pidfile($pid)
    $SIG{TERM} = \&do_clean_shutdown();
    $SIG{HUP}  = \&reload_config();
    $SIG{CHLD} = 'IGNORE';
    &run_main();
}
else { croak "fork failed: $!" }

.. Watch ..

We can check our children from time to time, how they are:

foreach my $child(@children) {
    if (kill 0 => $child ) {
        print "Child $pid is well and alive\n";
    }
}

Or can remove apparent heirs :

foreach my $child(@children) {
    print "Removing $pid from binary gen-pool\n";
    kill 'TERM' => $child;
    # maybe wait for the reaper?
}

... and Reap

We setup a routine that is called in case a child dies:

use POSIX ":sys_wait_h";
sub reaper {
    my $pid;
    while (($pid = waitpid(-1, WNOHANG)) > 0) {
        my $status=$?;
        $log_sys->info( "child $pid gone: $status");
        if (!$running) {
            unlink_pidfile($pid); # cleanup
        } else {
            fork_new();
        }
        $SIG{CHLD} = \&reaper;
    }
}

Slippery when wet

Signals (no matter if handled) are shiny .. or not. here some (ancient) code to retry open if it failed due to an occured interrupt:

my $maxtry=10;
do {
  $err = open($fh, '<', $locked_file);
  $errstr = $!;
  $maxtry--;
} while ($errstr eq "Interrupted system call"
        && $maxtry);

Caveats

Signal handling has some caveats, more or less serious depending on what you need, expect, or where you run your code.

• ALRM: Only one possible, maybe overwritten
• Signals may be deferred (safe?)
• Code may be restarted
• Everything may depend on anything (kernel, libs, perl, code)
• You got the signal, but does your code know?
• be Paranoid (do as little as possible) or Pragmatic (do anything necessary, it's better than doing nothing)?

plus lucis

Perls internal safe signal handling can be bypassed

• an ALARM may be deferred and trigger (much) too late
• instead of

  local $SIG{ALRM} = sub { die "alarm" };

• wie can access the signal by

  use POSIX qw(SIGALRM);
  POSIX::sigaction(SIGALRM,
        POSIX::SigAction->new(sub { die "alarm" }))
        or die "Error setting SIGALRM handler: $!\n";

• this may end in a core dump

Signals with Flags

We can additionally configure the flags for the signal handler:

• use Sys::SigAction qw( set_sig_handler );
  my $h = set_sig_handler( 'TERM' ,
      sub { ..... } , {
      flags => [ SA_RESTART ], # see signal.h
      mask  => [ qw( INT USR1 ), # 5.8.2+
      safe  => 1 }
  );

• Perl may retry interrupted calls before delivering them to the signal handler

Resources

Where to start, where to go after that
• perldoc perlipc
• perldoc POSIX
• Sys::SigAction
• Perl Cookbook Chapter 16 §13ff

Grey Lights?

Remember: mommy told you to look both ways before crossing the street!

And she was right: there might always be something to hit you in the back!

Thank you!