Fuzzing Irssi with Perl Scripts

TL;DR: Breaking tech news from Unnamed Source.

What Happened

📰 Unnamed Source is reporting on this story. This is a tech development worth watching closely.

Why It Matters

This story could have significant implications for the global community following tech trends.

Key Takeaways

• 📌 Reported by Unnamed Source
• 📌 Category: tech
• 📌 Read full story →

Follow GlobalWFeed on Telegram →

🤖 Automatically posted by Global Feed Bot

The task title reminded me of the song Let's Dance to Joy Division. Oddly applicable lyrics for a blog about programming: "But I worked something out last night that changed this little boy's brain, a small piece of advice that took 22 years in the make, and I will break it for you now. Please learn from my mistakes." And who doesn't love a wombat?

Task Description

You are given a list and a non-negative integer, $n. Write a script to divide the given list into $n equal parts. Return -1 if $n is more than the size of the list.

Example 1

Input: @list = ( 1,2,3,4,5), $n = 2
Output: ((1,2,3), (4,5))

• 5 / 2 = 2 remainder 1. The extra element goes into the first chunk.

Example 2

Input: @list = (1,2,3,4,5,6), $n = 3
Output: ((1,2), (3,4), (5,6))

• 6 / 3 = 2 remainder 0.

Example 3

Input: @list = (1,2,3), $n = 2
Output: ((1,2), (3))

Example 4

Input: @list = (1,2,3,4,5,6,7,8,9,10), $n = 5
Output: ((1,2), (3,4), (5,6), (7,8), (9,10))

Example 5

Input: @list = (1,2,3), $n = 4
Output: -1

Example 6

Input: @list = (72,57,89,55,36,84,10,95,99,35), $n = 7;
Output: ((72,57), (89,55), (36,84), (10), (95), (99), (35))

Elaboration

There's some simple modulo division here, and then a decision about what to do if it doesn't divide evenly into $n chunks. We could put the remainder into a chunk of its own, ignore the remainder, or we could distribute the leftovers in some way that feels balanced -- here, the specification implies that the bigger divisions should go at the front of the list.

Handling an invalid input (where $n is too big) by returning -1 instead of an array is a code smell. Having a function that returns different types of things is awkward to handle and error-prone. It would be more consistent to either throw an exception, or return an out-of-range value of the same type, such as an empty array or a null reference. But that wanker Kevin with his six months of Javascript experience got promoted to project manager for some reason <cough/>nepotism<cough> and that's the requirement we have.

This task has six examples, and usually we're given five. Is there some significance to this? Example 6 doesn't seem to add any new information beyond what we've already been given. But then we notice that the numbers could be ASCII characters, spelling "H9Y7$T?_c#". I'm pretty sure this is Kevin's password. Watch your back, Kevin.

To the CodeMobile, Robin!

sub listDivision($list, $n)
{
    my $size = @$list;
    return -1 if $n > $size;

    my $chunk = floor($size / $n);
    my $extra = $size % $n;
    my @answer;

    while ( $extra-- ) { push @answer, [ splice(@$list, 0, $chunk + 1) ]; }
    while ( @$list   ) { push @answer, [ splice(@$list, 0, $chunk    ) ]; }
    return \@answer;
}

The input parameters are an array and a number, which forces us to confront the way that Perl passes arguments as a flat list. Here, I've chosen to pass an array reference for $list instead of an array, which accomplishes two things: it removes the flat list problem to make it obvious which parameter is $n, and it's more efficient than passing a copy of the array. I could have also passed the parameters in the other order, but "project manager" Kevin said according to ChatGPT and Claude it had to be in this order, so yeah, that's that then.

Time to be serious. We dispense with the special case of returning -1 (more on that later). Then we calculate the quantities involved, and set up an array for producing the answer, initially empty.

The answer has two segments: a head part where there might be one extra element, and the tail part where the segments are of the calculated chunk size. One while loop ticks off the extra-sized chunks, and the second while loop completes the division.

To grab a group of contiguous elements from a list, the splice function seems like the obvious thing to do. Each splice is surrounded with [] so that the chunks are treated as array references. An alternative would have been to take an array slice, but that would introduce index variables for the range being sliced, which looks and feels more complicated and error-prone.

splice reduces the size of the list every time it's called, so the second while loop can use the size of $list as its condition. However, we are destroying $list as we go, so the calling function needs to be aware that the array it passed in is no longer valid. If it were important to keep the original list, a copy would need to be made either from the caller, or near the beginning of listDivision().

Let's think a moment about how to use this function. When it's called it could return either an array of arrays (references actually), or it could return a scalar -1. Here's how I wanted to write the script to use this function, using -n as a command-line option to get that parameter:

use GetOpt::Long;
my $N = 1;
GetOptions("n:i" => \$N);
my $div = listDivision([@ARGV], $N);
say showAofA($div);

where showAofA() is a function that formats the array of arrays into a parenthesized list of comma-separated lists, as in the examples.

However, $div isn't always a reference; sometimes it's a scalar -1. That throws up an ugly error about trying to use a scalar value as a reference. The way to check what's behind a variable in Perl is to use the ref function. It will give a string identifying the variable type, or, for a scalar, it will give an empty string. In perl, an empty string counts as false. So, to satisfy the requirement, the script needs to test whether ref($div) is empty:

say ( ref($div) ? showAofA($div) : $div );

What I probably should have done is change listDivision so that it consistently returns an array reference, but for the invalid case, throws an error. Variations on try/catch have been available as modules in Perl for decades, but it finally (see what I did there?) became official in version 5.38. That simplifies the usage of the function (if you consider try/catch simpler, which some consider a debatable topic).

try          { say showAofA( listDivision([@ARGV], $N) ); }
catch ( $e ) { say "-1" }

To make the function throw an exception, we only need to change the early return from the function into a die:

sub listDivision($list, $n)
{
    my $size = @$list;
    die "N out of range (must be <= $size)" if $n > $size;
    # [ ... ]

That's cleaner, but it slightly complicates the unit testing, because now we have to test for exceptions. The incantation is:

sub runTest
{
    use Test2::V0;
    use Test2::Tools::Exception;
    # [Test cases that should work
    is( listDivision([1,2,3], 1), [[1,2,3]], "One chunk");
    # [ ... ]
    # Test things that throw exceptions
    like ( dies { listDivision( [1,2,3], 4) }, qr/out of range/);

The dies function returns the exception string that results from trying to execute a code block, and like does a pattern match against the expected exception.

And finally, to tidy up a loose end, here's the showAofA() function.

sub showAofA($ref)
{
    return '('. join(',', map { '('.join(',', $_->@*).')' } $ref->@*) .')';
}

Let's take a bunch of punctuation out of that so that we can see the structure:

return ( join( map { join } ) ref )

$ref is a reference to an array of references; hence, $ref->@* is de-referencing it to yield a list of array references.

Each of those array references is then transformed by map. Inside the map {} block, $_ refers to one of the inner arrays, so we de-reference that ($_->@*) to get a list of numbers, which we will join into a comma-separated string. Wrap that with a pair of parentheses, and out of the map comes the inner lists formatted as needed.

Now each of the inner lists is a string. The outer join inserts commas to make it a list of lists, and finally we wrap the whole thing in parentheses to complete the string.

Note: This method is designed for Linux systems where LD_PRELOAD is available.
AddressSanitizer (ASan) is a powerful tool for detecting memory corruption in C/C++ code. When developing Perl XS modules, you can integrate ASan support directly into your Makefile.PL to make debugging seamless.

Add the 'asan-on-linux' Option

First, use GetOptions to provide a dedicated flag for ASan. This allows users to enable memory sanitization explicitly. We also set up a temporary directory to store ASan's detailed log files.

use Getopt::Long;
use File::Path 'mkpath';

GetOptions('asan-on-linux' => \my $asan_on_linux);

my @ccflags;
my @ldflags;
my $asan_logs_dir = ".tmp/asan_logs";

if ($asan_on_linux) {
    push @ccflags, "-fsanitize=address", "-fno-omit-frame-pointer";
    push @ldflags, "-fsanitize=address";
    
    mkpath $asan_logs_dir unless -d $asan_logs_dir;
}

Automate Library Preloading via Makefile Macros

Since the standard perl binary isn't typically compiled with ASan, we must ensure libasan.so is correctly preloaded before the interpreter starts. By using the macro parameter in WriteMakefile, we can automate this process for make test.

use Config;

WriteMakefile(
    NAME              => 'My::Module',
    CCFLAGS           => "$Config{ccflags} " . join(' ', @ccflags),
    LDDLFLAGS         => "$Config{lddlflags} " . join(' ', @ldflags),
    # ...
    macro => {
        $asan_on_linux ? (
            'override FULLPERL' => qq|LD_PRELOAD=\$\$($Config{cc} -print-file-name=libasan.so) ASAN_OPTIONS="log_path=$asan_logs_dir/asan.log:exitcode=0" $^X|
        ) : (),
    },
);

How to Handle Memory Issues

When ASan is enabled, the behavior depends on the type of memory issue:

• Memory Corruption: Issues like buffer overflows or use-after-free will cause a Segmentation Fault (Segfault) immediately. When this happens, you must fix the code until all tests pass without crashing.
• Memory Leaks: Unlike corruption, memory leaks do not cause a crash. To find them, you must check the logs after running your tests.

ASan generates a separate log file for each process. To efficiently find both corruption reports and leaks related to your specific module, use the following Perl one-liner (Replace My::Module with your module's name):

perl -00 -ne 'print "--- File: $ARGV ---\n$_\n" if /My::Module/' .tmp/asan_logs/* > .tmp/asan_summary.log

This command uses Perl's "paragraph mode" (-00) to extract the entire stack trace, allowing you to see exactly where the issue (crash or leak) was triggered in your native code.

Appendix: Platform Compatibility

macOS (Untested Hint)

On macOS, you might try DYLD_INSERT_LIBRARIES instead of LD_PRELOAD. However, System Integrity Protection (SIP) often prevents preloading into system binaries like /usr/bin/perl. You would likely need to use a perl installed via perlbrew or plenv.

Windows (Not Supported)

This mechanism is not applicable to Windows. Windows lacks LD_PRELOAD, and memory sanitization requires a fundamentally different configuration, typically involving a specially compiled perl interpreter.

Author: Yuki Kimoto

Weekly Challenge 373

Each week Mohammad S. Anwar sends out The Weekly Challenge, a chance for all of us to come up with solutions to two weekly tasks. My solutions are written in Python first, and then converted to Perl. Unless otherwise stated, Copilot (and other AI tools) have NOT been used to generate the solution. It's a great way for us all to practice some coding.

Challenge, My solutions

Task 1: Equal List

Task

You are given two arrays of strings.

Write a script to return true if the two given array represent the same strings otherwise false.

My solution

For input from the command line, I take a string with items concatenated by commas to generate the two lists. This seems to the easiest way to handle this, allowing for empty items as per the fourth example.

def main():
    result = equal_list(sys.argv[1].split(","), sys.argv[2].split(","))
    print("true" if result else "false")

This function is a one liner. I join the items in each list and then compare them to see if they are the same.

def equal_list(arr1: list[str], arr2: list[str]) -> bool:
    return ''.join(arr1) == ''.join(arr2)

The Perl solution has the same functionality.

sub main ( $str1, $str2 ) {
    my @arr1 = split /,/, $str1;
    my @arr2 = split /,/, $str2;

    say join( "", @arr1 ) eq join( "", @arr2 ) ? "true" : "false";
}

Examples

$ ./ch-1.py "a,bc" "ab,c"
false

$ ./ch-1.py "a,bc" "ab,c"
true

$ ./ch-1.py "a,b,c" "a,bc"
true

$ ./ch-1.py "a,bc" "a,c,b"
false

$ ./ch-1.py "ab,c," ",a,bc"
true

$ ./ch-1.py "p,e,r,l" "perl"
true

Task 2: List Division

Task

You are given a list and a non-negative integer.

Write a script to divide the given list into given non-negative integer equal parts. Return -1 if the integer is more than the size of the list.

My solution

For this task, I use the divmod function to calculate the number of items from the list each part needs, and the number of parts that require an extra item. These are stored as the variables item_length and extra_first.

I also have a variable called pos to store the position of the first letter. I have a loop i that runs n times. If i is less than extra_first, I take item_length + 1 items, otherwise item_length times, adding to pos as I go.

def list_division(input_list: list[int], n: int) -> list[list[int]] | None:
    result = []
    pos = 0

    if len(input_list) < n:
        return None

    item_length, extra_first = divmod(len(input_list), n)

    for i in range(n):
        this_length = item_length + 1 if i < extra_first else item_length
        result.append(input_list[pos:pos+this_length])
        pos += this_length

    return result

The Perl solution is similar. It doesn't have the pos variable, as it uses the splice function to remove the necessary number of items from the array. This function returns the items removed.

sub main (@list) {
    # The last value is the 'n' value
    my $n = pop(@list);
    my $length = scalar(@list);

    my @result = ();

    if ( $length < $n ) {
        say -1;
        return;
    }

    my $item_length = int( $length / $n );
    my $extra_first = $length % $n;

    foreach my $i ( 1 .. $n ) {
        my $this_length = $i <= $extra_first ? $item_length + 1 : $item_length;
        push @result, "(" . join( ",", splice( @list, 0, $this_length ) ) . ")";
    }

    say "(" . join( ", ", @result ) . ")";
}

Examples

$ ./ch-2.py 1 2 3 4 5 2
((1,2,3), (4,5))

$ ./ch-2.py 1 2 3 4 5 6 3
((1,2), (3,4), (5,6))

$ ./ch-2.py 1 2 3 2
((1,2), (3))

$ ./ch-2.py 1 2 3 4 5 6 7 8 9 10 5
((1,2), (3,4), (5,6), (7,8), (9,10))

$ ./ch-2.py 1 2 3 4
-1

$ ./ch-2.py 72 57 89 55 36 84 10 95 99 35 7 
((72,57), (89,55), (36,84), (10), (95), (99), (35))

This post explains the beginnings of my involvement with the Perl Tk debugger.

Origins

There are a number of well-known schisms in the world of computing:

• Emacs vs Vi
• Tabs vs Spaces
• Perl vs Python
• CLI vs GUI
• Windows vs everything else

I’d like to add another one: Debuggers vs Print Statements.

I come down on the side of debuggers. I’m a visual person and I’m impatient. I don’t like to wait for results. I like to get in there, crawl around, change things, see what’s going on.

I first started using debuggers on Digital Equipment Corporation VAX machines running VMS. I moved to Unix and used gdb in Emacs and then found ddd. Any time I started with a new language, one of the first things I reached for was a visual debugger.

When I started using Perl in the waning years of the last century, I conducted my usual search and found Devel::ptkdb. It served me well for all the years in which my primary language was Perl. Now, it did have its quirks. The variable display was less than ideal; there was no way to sort the variables in the vertically oriented display list. If you happened to have an array displayed before other variables, then as the array expanded, the other variables would scroll beyond the bottom of the window. The map of loaded modules sometimes did not properly find the related code. The map’s sort order was alphabetical and did not bring the most common modules, nor the local modules, to the top of the list where it would be more convenient to access them.

Although I might have tried to fix some of these issues, I was intimidated by the fact that this was a debugger, that it was a rather large piece of complicated software that interfaced with the command line Perl debugger. For what it did, it made my programming life much easier, and I was able to work around the quirks.

That all ended in 2009 when I changed jobs. Perl was no longer my primary environment. I ended up using a proprietary language written by a particular financial institution. There, I was not so intimidated. I enhanced that language’s debugger by adding a screen to manage breakpoints.

My next job was at a Java shop where I naturally learned how to use the Java debugger in what I think was the Eclipse ecosystem. At this shop, I was given a task to work on an old Perl program. The program read and wrote various files, but the file names were hard coded. They wanted me to make those names variable. In the process of making this change, I had to work on a production system because there was no dedicated development system. In order to make sure that I didn’t disturb production data, I was put on a read-only file system.

So now I had a chance to go back and work with Perl, and by extension, Devel::ptkdb. Once I had made my changes, I stepped through the program in the debugger in order to check them. Along the way, I set breakpoints and added variables in the variable display. Devel::ptkdb allows you to save the breakpoints and the variable list in a .ptkdb state file, which I did.

Imagine my surprise when I started up the program again in the debugger and my saved state was not loaded. I scratched my head for about 10 seconds and realized, “Ah, I’m on a read-only partition and the default location of the state file is the directory where the Perl script lives. That location is a read-only file system. Let me save the state file in my home directory.” ¹

Imagine my surprise when I started up the program again and my saved state did not reappear again. I scratched my head for a bit more than 10 seconds. I tried it again. It was still not saved. After about half a day of having to reset variables and breakpoints every time I ran the program, I decided that it was high time I took a look at the debugger code. I figured that it had to be something simple, related to just the input box for the name of the state file; it was not even a proper file dialog. I was sure I could handle that.

I was able to find the issue and correct it. By using the local corrected version of the debugger, I was able to save my state file to any writable directory.

It was at that point that something clicked in my head. If I could make that fix, maybe I could make other fixes. Maybe I could fix some of the quirks that bothered me all those years ago.

About two months and a huge changelog later, I had fixed some of those quirks. I fixed the sort problem for the variables. I fixed the mapping problem to the loaded code. I even made some improvements. The debugger used to not reload the state file if you restarted the debugger. Now it did. I programmed in a separate widget that I could use to print messages from inside the debugger while I was debugging the debugger.

I then realized I probably shouldn’t keep this to myself, so I contacted the author, figuring that they would like to take these changes and update the code. This was approximately 2013, and the debugger hadn’t been touched since 1999. However, it turned out that the author had moved on and would prefer that I just took over the project. I agreed, but before I could talk to legal about trying to let this code out of the shop, I ended up leaving that job.

I left all these changes behind a corporate firewall.

And that’s where they stayed. Over the years, I had occasion to use the debugger again, but never quite had the ‘tuits to recreate all these changes.

Well, now I have the ‘tuits. I’m currently unemployed and I have committed to making a presentation at the next Perl and Raku Conference about the debugger. It’s time to roll up my sleeves, dig in and see if I cannot recreate what I did over ten years ago.

Footnotes

1. Which, all these years later begs the question, “If the file system was read only, how did you update the script you were supposed to be changing?” Let’s not let details get in the way of a good story, shall we?  ↩