Unclear on the Concept

A list of true-life Programming and Sysadmin FAILs

(Identifying data is obscured to protect the clueless.)

########
# Start of main
#
main();
#
# END of main
#####

sub main(){
    ...

I came across this in a configuration file; the intent is clear: to remind people that the strings following are interpreted as regular expressions, and thus '.' needs to be escaped: '\.'. I came across this looking at it in less, using 7-bit ASCII, and some UTF-8 made this unintentionally very funny:

# patterns are used as regex so special characters have to be escaped 
# (ex: <E2><80><98>.<E2><80><99> should be given as <E2><80><98>\.<E2><80><99>)

less shows non-printing characters as <XX> (XX are hexadecimal digits). E2 80 98 is the UTF-8 encoding for left quote, U+2018 (  ‘   — &#x2018; or &lsquo;) and E2 80 99 is right-quote, U+2019 (  ’   — &#x2019; or &rsquo;).

2014-05-09: Holy Redundant Redundancy, Pathman!

A few days ago, I discovered that some old code was archiving data into the following path:

/path/to/$HOSTNAME/2014-05/05/06/$FILENAME

At first, descending into the hierarchy, I thought that year and month were just jammed together at the same level, but then each %Y-%m had only one subdirectory (matching %m), and that subdirectory had subdirs ('01' through '30' or '31') instead of files. It was quite confusing.

We get one of these at least every 3 months from various places.

In this example, A.B.C.D is my company's IP address, and Q.R.S.T is the other company's IP address.

I have annotated the "WTF?" information with ^^^^^^^.

From: no-reply@abuse.$TLD [mailto:no-reply@abuse.$TLD]
Sent: Tuesday, March 25, 2014 20:33
To: hostmaster@$MY_COMPANY
Subject: Illegal activity from A.B.C.D

We have noticed suspicious activity from A.B.C.D aimed at one of our
servers.  Please investigate this host and disable whichever exploit
or malware is causing this activity.  For more information or
questions please refer to our website located at http://www.abuse.bz/

Here are our raw logs:
==
[2014-03-25 04:50:15 CET] [Timestamp: 1395719416] [10861574.834874]
Firewall: *UDP_IN Blocked* IN=eth0 OUT= SRC=A.B.C.D DST=Q.R.S.T
LEN=296 TOS=0x08 PREC=0x40 TTL=45 ID=0 DF PROTO=UDP SPT=53 DPT=50069 LEN=276
                                          ^^^^^^^^^^^^^^^^

ORIGINAL_DATE=`date +%Y%m%d`
NEW_DATE=$(date -d "1970-01-01 $(($(date -d "${ORIGINAL_DATE}" "+%s")  )) sec" "+%Y-%m-%d")

This is a very long way around to doing the same as

ORIGINAL_DATE=`date +%Y%m%d`
NEW_DATE=`date +%Y-%m-%d`

but it does have some lovely examples of bash syntactic sugar (both $(expression) [a replacement for `expr`, one which can be nested] and $((expression)), which is for arithmetic expressions).

#!/usr/local/bin/perl
@LIST=("name1","name2","name3","name4");
$ending_value=scalar(@LIST);

for ($counter=0;$counter<$ending_value;$counter++)
{
  system("/usr/local/bin/perl /path/to/bin/loging/logarchiver.pl $LIST[$counter]");
}

NOTE: that "loging" is actually what is there in the filesystem.

2013-06-06: This option we tell you about is illegal.

solaris% /usr/sfw/sbin/snmpd --version
snmpd: illegal option -- version

Usage:  snmpd [OPTIONS] [LISTENING ADDRESSES]

        Version:  5.0.9
        Web:      http://www.net-snmp.org/
        Email:    net-snmp-coders@lists.sourceforge.net

  -a                    log addresses
  -A                    append to the logfile rather than truncating it
  ...<SNIP>...
  -v, --version         display version information
  ...<SNIP>...

        if( $line =~ /^([Tt][Oo]:|[Cc][Cc]:).*/ ) {
           next;

        } elsif( $line =~ /^([Ff][Rr][Oo][Mm]:|[Rr][Ee][Pp][Ll][Yy]-[Tt][Oo]:|[Rr][Ee][Tt][Uu][Rr][Nn]-[Pp][Aa][Tt][Hh]:)(.*)/ ) {

And don't even get me started on the formatting horror of if(...) instead of if (...).

I work with a project manager who codes project spreadsheet rows by "Completed", "Behind", "Needs Attention", and "On Target". So far, so good. $PROJECT_MANAGER also color-codes them, in a way that looks like the German or Bolivian flag:

Again: so far, so good. The colors are fairly obvious.

But then $PROJECT_MANAGER a separate column for each row to show status. And instead of including "Completed", "Behind", "Needs Attention", and "On Target" text, in that column, $PROJECT_MANAGER uses "RYGC":

It took me several minutes to realize that the "RYGC" tags were just color name abbreviations.

A coworker is trying to get help from $VENDOR. Good luck with that.

Latest update to my $PROBLEM case with $VENDOR:

Before we go any further with this, I need to point out that sudo is
not supported by $VENDOR on $OPERATING_SYSTEM.  Can you try the
command actually being root, rather than via sudo?

Found in one of my own scripts. These two clauses follow one-on-the-other. Note that $DEBUG is an integer.

if ( $DEBUG > 2 ) {
    # ... bunch of debugging ...
}

if ( $DEBUG >= 3 ) {
    # ... bunch more debugging ...
}

Sigh.

Possibly the worst thing I've seen in a perl script in a long time:

    @{$ip} = (@{$ip},$line);

1. $ip is being used as a symbolic reference, @{$ip} is an array (in the current namespace) named the value of $ip (ugh)
2. it re-implements push() in an incredibly inefficient way

I just realized that my company has the following three scripts, all called sequentially from our big log parsing script:

parse-outbound-${DAEMON}-logs
parse_${DAEMON}_outbound
parse-${DAEMON}-outbound

Note the underscores (_) versus hyphens (-).

We discovered that we have a router that drops any packet whose IP header checksum is 0xffff. So for some subset of host pairs (whose source and destination IP addresses are in just the wrong range), one particular packet length will lead to packets being dropped. (This is not a big deal for TCP, because the IP identification field increments upon retry, but UDP does not retry, so certain things simply won't work with UDP.)

From an email message after someone traced data flow:

"There is a PL/SQL package in $DATABASE that this CGI script calls indirectly (CGI script executes ssh, which in turn executes sqlplus on $DATABASE_HOST, which in turn executes $SQLSCRIPT.sql script which in turn calls $STORED_PROCEDURE_NAME procedure)."

I just spent far too long diving through far too many perl scripts that use functions from a "Pushpull.pm" module. One of which is called push. So to avoid colliding with the built-in push, you have to call it by &push(...). Now I understand why all that code looked like it was from the Perl 4 era....

Kill. Me. Now.

% ls -l /etc/logcheck/ignore.d.server/exim
lrwxrwxrwx    1 root     logcheck       23 Jan 28 06:04 /etc/logcheck/ignore.d.server/exim -> ../ignore.d.server/exim

This stupidity is not our company; our email servers can't send email to them for some reason:

% host example.com
example.com is an alias for example.com.
example.com is an alias for example.com.
example.com is an alias for example.com.

(No, the domain in question was not actually example.com.)

I found a script today that has a lock file, which is only referenced when creating it and removing it. They even have special code to remove it when it gets a SIGINT. But nothing actually tests for its existence.

$lock_file   = "/path/to/lockfile";

$SIG{'INT'}  = 'INT_handler';

#-- Create lock file before processing

`touch $lock_file`;

#--Remove Lock file after processing

unlink $lock_file;
exit(0);

sub INT_handler {

        print  STDERR "Program Interrupted\n";

        if( -f $lock_file) {
                unlink $lock_file;
        }
        exit(-2);

}

Found in a shell script, repeated a bunch of times:

echo "Error: code $? when transfering $FILE, skip" |tee|logger -t $tag -p $priority

tee without any arguments simply copies its standard input to its standard output: it's basically cat here, which is useless.

Found in a perl script:

my $end_time         = `perl -e '{print time(), "\n"}'`;

This isn't really an "unclear on the concept" — it's just very funny.

-----Original Message-----
From: Cron Daemon [mailto:root@HOST]
Sent: Thursday, March 18, 2010 16:31
To: root@HOST
Subject: Cron <root@HOST> [ -x /usr/local/sbin/SCRIPT ] &&  /usr/local/sbin/SCRIPT 2>&1

/var/log/faillog: No such file or directory

This is from old monitoring code. Server is a socket. $timeout = 5. I found this when trying to figure out why this always paused for an extra 5 seconds. I include my fix as a hint as to why this is dumb.

-    while (select($socketvecout=$socketvec, undef, undef, $timeout) ) {
+    if (select($socketvecout=$socketvec, undef, undef, $timeout)) {
         recv(Server, $server_reply, 1500, 0)
             or die "udp recv failed testing $port: $!\n";
     }
+    else {
+        return "timed out waiting for response from $daemon on port $port";
+    }

Today I ran across a script with two fun examples.

First example:

DATE=`perl -e '@arr = localtime(time - 86400); printf "%04d:%02d:%02d", $arr[5]+1900, $arr[4]+1, $arr[3];'`

YEAR=`echo $DATE | cut -d: -f1`
MONTH=`echo $DATE | cut -d: -f2`
DAY=`echo $DATE | cut -d: -f3`

This one is not totally insane, just ugly and inefficient. There are a number of ways to fix it, including a single 'eval' line:

eval `perl -e '@arr = localtime(time - 86400); printf "YEAR=%04d; MONTH=%02d ; DAY=%02d", $arr[5]+1900, $arr[4]+1, $arr[3];'`

What's interesting is that they only need $YEAR, $MONTH, and $DAY separate because of the other example from this script:

if [ ! -d $ARCHIVE_DIR/$YEAR ]
then
        mkdir -p $ARCHIVE_DIR/$YEAR
fi

if [ ! -d $ARCHIVE_DIR/$YEAR/$MONTH ]
then
        mkdir -p $ARCHIVE_DIR/$YEAR/$MONTH
fi

if [ ! -d $ARCHIVE_DIR/$YEAR/$MONTH/$DAY ]
then
        mkdir -p $ARCHIVE_DIR/$YEAR/$MONTH/$DAY
fi

'mkdir -p' already makes any missing parent directories, and doesn't make (nor throw an error for) any directory that already exists), so this can be collapsed to

mkdir -p $ARCHIVE_DIR/$YEAR/$MONTH/$DAY

And thus both examples could be collapsed to two lines:

YMD=`perl -e '@arr = localtime(time - 86400); printf "%04d/%02d/%02d", $arr[5]+1900, $arr[4]+1, $arr[3];'`
mkdir -p $ARCHIVE_DIR/$YMD

and then from then on, $YMD could be used (because the script refers only to $YEAR/$MONTH/$DAY [combined] from then on).

What, you mean Perl's print doesn't convert strftime() conversion specifiers for you automatically?

print $sendmail_fh "Date: %a, %d %b %Y %T %Z\n";

(This is printing to a pipe connected to the standard input of /usr/lib/sendmail -t. This is also my own error. Sigh.)

% ps -ef | grep "[t]ail"
clueless 25624   924   0   Jul 17 ?   0:00 tcsh -c tail -f /path/to/logfile | tail -3
clueless 25660 25624   0   Jul 17 ?   5:07 tail -f /path/to/logfile
clueless 25661 25624   0   Jul 17 ?   0:03 tail -3

I came across a perl script today that was entirely made up of

system("shell-command1 ARGS1");

system("shell-command2 ARGS2");

system("shell-command3 ARGS3");

and so on. Sigh.

sub getHashCount($){
  my $count = 0;
  for my $k (keys %{$_[0]}){
    $count++;
  }
  return $count;
}

(As opposed to simply using

scalar(keys %hash)

% head -6 /path/to/logroll/script
# daemon's agelogs script
#
#!/bin/sh

# $Source: /path/to/cvs/repository/for/logroll/script,v $
# $Id: script,v 1.4 2008/03/11 17:38:35 clueless Exp $
%

It's a darn good thing that /bin/sh is the default shell.

2008-03-03: Leap Before You Look (and Stupid Remediation)

From: $PERSON [mailto:$USERNAME@$COLO_VENDOR.com]
Sent: Monday, March 03, 2008 11:24 AM
To: $CONTACT_AT_OUR_COMPANY
Subject: $COLO_VENDOR HVAC Outage

Explanation of HVAC problem

$COLO_VENDOR utilizes a chilled water system comprised of three 240 ton
chillers feeding 20 computer room air conditioners (CRAC). The system is
under the control of a building management system (BMS) which maintains the
chillers in a Lead-Lag-Standby configuration that rotates monthly.
Unfortunately, when the system was set up, the programmer overlooked adding
the provision for Leap Year day. At midnight of the 28th, the BMS rotated
the lead-lag-standby sequence, but since the system wasn't programmed to
recognize Feb 29th, all the chillers shutdown, waiting for March 1st to
resume operation. About 1:30 am high temperature alarms were received.
Technicians and management responded and began bringing the HVAC back on
line as well as responding to numerous customer phone calls.

To prevent this problem from recurring, the vendor for the BMS system will
reprogram the system to recognize Feb 29th, and alarms will be added to the
system to immediately report when all chillers are off line.

The initial problem is bad enough, but for the fix — wouldn't it be simpler to have it start the lead-lag-standby rotation starting at 00:01 on the first of the month? Every month has a day 1.

See Also

• Useless Use of 'cat' Award

$Id: unclear-on-the-concept.html,v 1.52 2023/02/23 14:39:22 jdpaul Exp $
$Source: /home/jdpaul/websrc/RCS/unclear-on-the-concept.html,v $