Introduction

In this multipart series we will explore the benefits (and pitfalls) of vibe coding a Perl interface to an external (or foreign) library through a large language model.
Those of you who follow me on X/Twitter (as @ChristosArgyrop and @ArgyropChristos),
Bluesky , mast.hpc, mstdn.science,
mstdn.social know that I have been very critical of the hype behind AI and the hallucinations of both models and the
human AI influencers (informally known as botlickers in some corners of the web).

However, there are application areas of vibe coding with AI, e.g. semi-automating the task of creating API from one one language to another, in which the chatbots may actually deliver well and act as productivity boosters.
In this application area, we will be leveraging the AI tools as more or less enhanced auto-complete tools that can help a developer navigate less familiar, more technical and
possibly more boring aspects of the target language's 'guts'. If AI were to deliver in this area, then meaningless language wars can be averted, and wrong (at least in my opinion) reasons
to prefer one language, i.e. the availability of some exotic library, may be avoided.

For my foray in this area, I chose to interface to the library Bit that I wrote to support text fingerprinting for some research
applications. The library based on David Hanson's Bit_T library discussed in
Chapter 13 of "C Interfaces and Implementations"
has been extended to incorporate additional operations on bitsets (such as counts on unions/differences/intersections of sets) and fast population counts in both CPU and GPU
(with TPU implementations coming down the road). Hence, this is a test case that can illustrate the utility of Perl in using code that executes transparently in various hardware assets.
Similar to Hanson's original implementation (after all, my work is based on his!) the library interface is implemented to
an Abstract Data Type in C; a crucial aspect of the implementation is to manage memory and avoid leaks without looking (at all, or as little as possible!) under the hood.
For our experiments we will use Claude 3.7 Thinking Sonnet through the Github Copilot vscode interface. This is going to be a multipart series post that will be published throughout the summer.
Our focus will be on interactions between me and the bot, and in particular critiquing the responses it has given me for both high level (e.g. choice of approach) and technical aspects of the project.

The Prompt

For the prompt I provided the "bit.c", "bit.h", makefile and the readme file of the Bit github repository as context to Claude and then I issued the following:

Look at the bit.h file that defines two abstract data types Bit_T and Bit_T_DB
and their interfaces in C. I would like to wrap these interfaces in Perl.
The build process will  consist of using the Alien package in Perl to make bit available to Perl.
Explore possible options for the creation of the interface: 
A. Creation of the bit  C library :
1) compiling bit.c and bit.h as dynamic libraries
2) compiling bit.c nd bit.h into a static library
B. Interfacing with the bit C library from Perl:
1) using SWIG i files
2) using Platypus::FFI
3) using Inline::C
4) using XS

Since you are an expert with deep knowledge of the Perl ecosystem, consider pros and cons of
each approach. and then recommend the optimal Consider lightweight options for encapsulating
the Bit_T and Bit_T_DB abstract data types as Perl objects.
Provide wrapping recommendations targetting a beginner, intermediate and experienced Perl
programmer since those may differ.  When providing recommendations , consider the dependencies
that are required for each implementation and highlight them in your analysis.
Once you have provided the recommendation, provide the implementation of the wrapper for
these 3 programmer levels. To reduce the token output (and your risk of hallucinating),
provide a full implementation of the wrapper for the beginner level and only the
changes (if any) that are required for the intermediate and advanced programmer levels.

In my prompt specification, I clearly specified that

• linkage to the foreign library could be either as a static or a dynamic object,
• the desired way to package the dependency for use in Perl is an Alien package
• all four possible ways to interface to a foreign library from Perl should be considered.
• asked it to consider the problem of wrapping from the perspective of Perl developers with different levels of expertise.

Note that I gave the bot considerable freedom over the "low" level aspects of wrapping (type of library, choice of the cross-language interface),
but restricted it in terms of how I wanted the library served.
Furthermore, I guided it to consider alternative viewpoints and level of expertise, providing a unique opportunity to test Claude's
representation¹ of conversations about wrappers. Finally, I alerted the chatbot to consider the token-sequence length dependent risk of hallucinations and
tried to limit the length of the output of the alternative implementations.

Claude's Response

The output based on this prompt is shown below (inside the horizontal lines); the formatting was also part of the output

Perl Wrapper for Bit_T and Bit_T_DB C Library

Let me analyze the options for creating a Perl interface to the Bit_T and Bit_T_DB C library and provide implementation recommendations for programmers at different experience levels.

A. Creation of the Bit C Library

1. Dynamic Library Approach

I see the Makefile already creates a dynamic library (libbit.so) in the build directory with this rule:

$(TARGET): $(OBJ)
    $(CC) $(CFLAGS) -shared -o $@ $^ $(LDFLAGS)

Pros:

• Runtime binding, reducing memory footprint
• Updates to the library don't require recompiling Perl code
• Already implemented in your build system
• Better for distributing the library separately from the wrapper

Cons:

• Deployment requires ensuring the library is in the system path
• Potential version compatibility issues

2. Static Library Approach

To compile as a static library, you would modify the Makefile:

TARGET = $(BUILD_DIR)/libbit.a

$(TARGET): $(OBJ)
    ar rcs $@ $^

Pros:

• Self-contained distribution
• No runtime dependency resolution
• Potentially faster function calls
• No path issues during deployment

Cons:

• Larger binary size
• Need to recompile wrapper if library changes
• Less memory efficient if multiple programs use the library

B. Perl Interfacing Options

1. SWIG

Pros:

• Automated wrapper generation
• Supports multiple languages (future portability)
• Good for complex interfaces

Cons:

• Learning curve for SWIG interface files
• Additional build dependency
• Complex for beginners

2. FFI::Platypus

Pros:

• No compilation required at installation time
• Modern, actively maintained
• Cleaner code than XS
• Suitable for your library structure

Cons:

• Requires FFI::Platypus and related modules
• Less efficient for very complex interfaces

3. Inline::C

Pros:

• Simple integration of C code directly in Perl
• No separate interface files
• Good for prototyping

Cons:

• C compiler required at runtime
• Less suitable for large libraries
• Performance overhead for initialization

4. XS

Pros:

• Standard Perl way to interface with C
• Excellent performance
• Tight integration with Perl's memory management

Cons:

• Steep learning curve
• Verbose, low-level coding
• More complex build process

Recommendation for Each Level

Beginner Level: FFI::Platypus with Dynamic Library

FFI::Platypus provides the easiest approach for beginners with minimal C knowledge. It works with your existing dynamic library build.

Intermediate Level: Inline::C with Dynamic Library

Inline::C offers more control for intermediate users who understand some C code but want to avoid complex build processes.

Advanced Level: XS with Static Library

XS provides the highest performance and greatest control, suitable for experienced Perl programmers.

Implementation for Beginner Level: FFI::Platypus

Let's create the Alien::Bit module first to handle library discovery, then implement Bit::Set and Bit::Set::DB modules using FFI::Platypus.

Step 1: Create Alien::Bit module

# /media/chrisarg/Software-Dev/Bit/perl-wrapper/lib/Alien/Bit.pm
package Alien::Bit;

use strict;
use warnings;
use parent 'Alien::Base';
use File::Spec;

sub dynamic_libs {
    my $class = shift;

    # Try to find the library in common locations
    my @potential_paths = (
        # From the build directory
        '/media/chrisarg/Software-Dev/Bit/bit/build/libbit.so',

        # System paths
        '/usr/local/lib/libbit.so',
        '/usr/lib/libbit.so',

        # Environment variable if set
        $ENV{BIT_LIB_PATH} ? File::Spec->catfile($ENV{BIT_LIB_PATH}, 'libbit.so') : ()
    );

    foreach my $path (@potential_paths) {
        return $path if -f $path && -r $path;
    }

    die "Could not find libbit.so library. Please set BIT_LIB_PATH environment variable to its location.\n";
}

sub include_dirs {
    return (
        '/media/chrisarg/Software-Dev/Bit/bit/include',
        '/usr/local/include',
        '/usr/include',
        $ENV{BIT_INCLUDE_PATH} || ()
    );
}

1;

Step 2: Create Bit::Set module for individual bitsets

# /media/chrisarg/Software-Dev/Bit/perl-wrapper/lib/Bit/Set.pm
package Bit::Set;

use strict;
use warnings;
use FFI::Platypus;
use FFI::Platypus::Memory qw(malloc free);
use FFI::Platypus::Buffer qw(scalar_to_buffer buffer_to_scalar);
use Alien::Bit;

# Create FFI::Platypus object
my $ffi = FFI::Platypus->new(api => 1);

# Add path to our dynamic library
$ffi->lib(Alien::Bit->dynamic_libs);

# Define opaque types for our bitset pointers
$ffi->type('opaque' => 'Bit_T');
$ffi->type('opaque*' => 'Bit_T_Ptr');

# Define our Bit_T functions
$ffi->attach(Bit_new => ['int'] => 'Bit_T' => sub {
    my ($xsub, $self, $length) = @_;
    die "Length must be a positive integer" unless defined $length && $length > 0;
    my $bit_set = $xsub->($length);
    die "Failed to create bit set" unless $bit_set;
    return bless { _handle => $bit_set }, $self;
});

$ffi->attach(Bit_free => ['Bit_T_Ptr'] => 'opaque' => sub {
    my ($xsub, $self) = @_;
    my $ptr = \$self->{_handle};
    return $xsub->($ptr);
});

$ffi->attach(Bit_length => ['Bit_T'] => 'int' => sub {
    my ($xsub, $self) = @_;
    return $xsub->($self->{_handle});
});

$ffi->attach(Bit_count => ['Bit_T'] => 'int' => sub {
    my ($xsub, $self) = @_;
    return $xsub->($self->{_handle});
});

$ffi->attach(Bit_buffer_size => ['int'] => 'int');

$ffi->attach(Bit_bset => ['Bit_T', 'int'] => 'void' => sub {
    my ($xsub, $self, $index) = @_;
    die "Index must be non-negative" unless defined $index && $index >= 0;
    $xsub->($self->{_handle}, $index);
});

$ffi->attach(Bit_bclear => ['Bit_T', 'int'] => 'void' => sub {
    my ($xsub, $self, $index) = @_;
    die "Index must be non-negative" unless defined $index && $index >= 0;
    $xsub->($self->{_handle}, $index);
});

$ffi->attach(Bit_get => ['Bit_T', 'int'] => 'int' => sub {
    my ($xsub, $self, $index) = @_;
    die "Index must be non-negative" unless defined $index && $index >= 0;
    return $xsub->($self->{_handle}, $index);
});

$ffi->attach(Bit_set => ['Bit_T', 'int', 'int'] => 'void' => sub {
    my ($xsub, $self, $lo, $hi) = @_;
    die "Low index must be non-negative" unless defined $lo && $lo >= 0;
    die "High index must be greater than or equal to low index" unless defined $hi && $hi >= $lo;
    $xsub->($self->{_handle}, $lo, $hi);
});

$ffi->attach(Bit_clear => ['Bit_T', 'int', 'int'] => 'void' => sub {
    my ($xsub, $self, $lo, $hi) = @_;
    die "Low index must be non-negative" unless defined $lo && $lo >= 0;
    die "High index must be greater than or equal to low index" unless defined $hi && $hi >= $lo;
    $xsub->($self->{_handle}, $lo, $hi);
});

# Comparison operations
$ffi->attach(Bit_eq => ['Bit_T', 'Bit_T'] => 'int' => sub {
    my ($xsub, $self, $other) = @_;
    die "Other bitset must be a Bit::Set object" unless ref $other eq ref $self;
    return $xsub->($self->{_handle}, $other->{_handle});
});

# Set operations
$ffi->attach(Bit_union => ['Bit_T', 'Bit_T'] => 'Bit_T' => sub {
    my ($xsub, $self, $other) = @_;
    die "Other bitset must be a Bit::Set object" unless ref $other eq ref $self;
    my $result_handle = $xsub->($self->{_handle}, $other->{_handle});
    return bless { _handle => $result_handle }, ref $self;
});

$ffi->attach(Bit_inter => ['Bit_T', 'Bit_T'] => 'Bit_T' => sub {
    my ($xsub, $self, $other) = @_;
    die "Other bitset must be a Bit::Set object" unless ref $other eq ref $self;
    my $result_handle = $xsub->($self->{_handle}, $other->{_handle});
    return bless { _handle => $result_handle }, ref $self;
});

# Count operations
$ffi->attach(Bit_inter_count => ['Bit_T', 'Bit_T'] => 'int' => sub {
    my ($xsub, $self, $other) = @_;
    die "Other bitset must be a Bit::Set object" unless ref $other eq ref $self;
    return $xsub->($self->{_handle}, $other->{_handle});
});

# Constructor and destructor
sub new {
    my ($class, $length) = @_;
    return $class->Bit_new($length);
}

sub DESTROY {
    my ($self) = @_;
    $self->Bit_free() if defined $self->{_handle};
}

# Convenient accessor methods
sub length {
    my ($self) = @_;
    return $self->Bit_length();
}

sub count {
    my ($self) = @_;
    return $self->Bit_count();
}

sub get {
    my ($self, $index) = @_;
    return $self->Bit_get($index);
}

sub set {
    my ($self, $index) = @_;
    $self->Bit_bset($index);
    return $self;
}

sub clear {
    my ($self, $index) = @_;
    $self->Bit_bclear($index);
    return $self;
}

sub set_range {
    my ($self, $lo, $hi) = @_;
    $self->Bit_set($lo, $hi);
    return $self;
}

sub clear_range {
    my ($self, $lo, $hi) = @_;
    $self->Bit_clear($lo, $hi);
    return $self;
}

sub equals {
    my ($self, $other) = @_;
    return $self->Bit_eq($other) ? 1 : 0;
}

sub union {
    my ($self, $other) = @_;
    return $self->Bit_union($other);
}

sub intersection {
    my ($self, $other) = @_;
    return $self->Bit_inter($other);
}

sub intersection_count {
    my ($self, $other) = @_;
    return $self->Bit_inter_count($other);
}

1;

Step 3: Create Bit::Set::DB module for packed containers

# /media/chrisarg/Software-Dev/Bit/perl-wrapper/lib/Bit/Set/DB.pm
package Bit::Set::DB;

use strict;
use warnings;
use FFI::Platypus;
use FFI::Platypus::Memory qw(malloc free);
use FFI::Platypus::Buffer qw(scalar_to_buffer buffer_to_scalar);
use Bit::Set;
use Alien::Bit;

# Create FFI::Platypus object
my $ffi = FFI::Platypus->new(api => 1);

# Add path to our dynamic library
$ffi->lib(Alien::Bit->dynamic_libs);

# Define opaque types for our bitset pointers
$ffi->type('opaque' => 'Bit_T');
$ffi->type('opaque' => 'Bit_T_DB');
$ffi->type('opaque*' => 'Bit_T_DB_Ptr');

# Define struct for SETOP_COUNT_OPTS
$ffi->type('record(SETOP_COUNT_OPTS)' => [
    num_cpu_threads => 'int',
    device_id => 'int',
    upd_1st_operand => 'bool',
    upd_2nd_operand => 'bool',
    release_1st_operand => 'bool',
    release_2nd_operand => 'bool',
    release_counts => 'bool'
]);

# Define Bit_T_DB functions
$ffi->attach(BitDB_new => ['int', 'int'] => 'Bit_T_DB' => sub {
    my ($xsub, $self, $length, $num_of_bitsets) = @_;
    die "Length must be a positive integer" unless defined $length && $length > 0;
    die "Number of bitsets must be a positive integer" unless defined $num_of_bitsets && $num_of_bitsets > 0;
    my $db = $xsub->($length, $num_of_bitsets);
    die "Failed to create bitset DB" unless $db;
    return bless { _handle => $db }, $self;
});

$ffi->attach(BitDB_free => ['Bit_T_DB_Ptr'] => 'opaque' => sub {
    my ($xsub, $self) = @_;
    my $ptr = \$self->{_handle};
    return $xsub->($ptr);
});

$ffi->attach(BitDB_length => ['Bit_T_DB'] => 'int' => sub {
    my ($xsub, $self) = @_;
    return $xsub->($self->{_handle});
});

$ffi->attach(BitDB_nelem => ['Bit_T_DB'] => 'int' => sub {
    my ($xsub, $self) = @_;
    return $xsub->($self->{_handle});
});

$ffi->attach(BitDB_count_at => ['Bit_T_DB', 'int'] => 'int' => sub {
    my ($xsub, $self, $index) = @_;
    die "Index must be non-negative" unless defined $index && $index >= 0;
    return $xsub->($self->{_handle}, $index);
});

$ffi->attach(BitDB_clear => ['Bit_T_DB'] => 'void' => sub {
    my ($xsub, $self) = @_;
    $xsub->($self->{_handle});
});

$ffi->attach(BitDB_clear_at => ['Bit_T_DB', 'int'] => 'void' => sub {
    my ($xsub, $self, $index) = @_;
    die "Index must be non-negative" unless defined $index && $index >= 0;
    $xsub->($self->{_handle}, $index);
});

$ffi->attach(BitDB_get_from => ['Bit_T_DB', 'int'] => 'Bit_T' => sub {
    my ($xsub, $self, $index) = @_;
    die "Index must be non-negative" unless defined $index && $index >= 0;
    my $bit_handle = $xsub->($self->{_handle}, $index);
    return bless { _handle => $bit_handle }, 'Bit::Set';
});

$ffi->attach(BitDB_put_at => ['Bit_T_DB', 'int', 'Bit_T'] => 'void' => sub {
    my ($xsub, $self, $index, $bitset) = @_;
    die "Index must be non-negative" unless defined $index && $index >= 0;
    die "Bitset must be a Bit::Set object" unless ref $bitset eq 'Bit::Set';
    $xsub->($self->{_handle}, $index, $bitset->{_handle});
});

# CPU-specific intersection count function
$ffi->attach(BitDB_inter_count_cpu => ['Bit_T_DB', 'Bit_T_DB', 'SETOP_COUNT_OPTS'] => 'int*' => sub {
    my ($xsub, $self, $other, $opts) = @_;
    die "Other must be a Bit::Set::DB object" unless ref $other eq ref $self;
    $opts ||= { 
        num_cpu_threads => 1, 
        device_id => -1, 
        upd_1st_operand => 0, 
        upd_2nd_operand => 0,
        release_1st_operand => 0,
        release_2nd_operand => 0,
        release_counts => 0
    };
    return $xsub->($self->{_handle}, $other->{_handle}, $opts);
});

# Constructor and destructor
sub new {
    my ($class, $length, $num_of_bitsets) = @_;
    return $class->BitDB_new($length, $num_of_bitsets);
}

sub DESTROY {
    my ($self) = @_;
    $self->BitDB_free() if defined $self->{_handle};
}

# Convenient accessor methods
sub length {
    my ($self) = @_;
    return $self->BitDB_length();
}

sub num_of_bitsets {
    my ($self) = @_;
    return $self->BitDB_nelem();
}

sub count_at {
    my ($self, $index) = @_;
    return $self->BitDB_count_at($index);
}

sub clear {
    my ($self) = @_;
    $self->BitDB_clear();
    return $self;
}

sub clear_at {
    my ($self, $index) = @_;
    $self->BitDB_clear_at($index);
    return $self;
}

sub get_bitset {
    my ($self, $index) = @_;
    return $self->BitDB_get_from($index);
}

sub put_bitset {
    my ($self, $index, $bitset) = @_;
    $self->BitDB_put_at($index, $bitset);
    return $self;
}

sub intersection_count_cpu {
    my ($self, $other, $opts) = @_;
    return $self->BitDB_inter_count_cpu($other, $opts);
}

1;

Step 4: Example usage script

#!/usr/bin/perl

use strict;
use warnings;
use Bit::Set;
use Bit::Set::DB;

# Create a new bitset with 1024 bits
my $bitset1 = Bit::Set->new(1024);
my $bitset2 = Bit::Set->new(1024);

# Set some bits
$bitset1->set(42);
$bitset1->set(100);
$bitset2->set(42);
$bitset2->set(200);

# Check if a bit is set
print "Bit 42 in bitset1: ", $bitset1->get(42), "\n";
print "Bit 100 in bitset1: ", $bitset1->get(100), "\n";
print "Bit 200 in bitset1: ", $bitset1->get(200), "\n";

# Count the number of bits set
print "Number of bits set in bitset1: ", $bitset1->count(), "\n";
print "Number of bits set in bitset2: ", $bitset2->count(), "\n";

# Calculate intersection count
my $intersection_count = $bitset1->intersection_count($bitset2);
print "Intersection count: $intersection_count\n";

# Create a new bitset as the union of the two bitsets
my $union = $bitset1->union($bitset2);
print "Number of bits set in union: ", $union->count(), "\n";

# Create a BitDB with 3 bitsets of length 1024
my $db = Bit::Set::DB->new(1024, 3);

# Put our bitsets into the DB
$db->put_bitset(0, $bitset1);
$db->put_bitset(1, $bitset2);
$db->put_bitset(2, $union);

# Count bits in each bitset in the DB
print "Bits set in DB at index 0: ", $db->count_at(0), "\n";
print "Bits set in DB at index 1: ", $db->count_at(1), "\n";
print "Bits set in DB at index 2: ", $db->count_at(2), "\n";

Modifications for Intermediate Level (Inline::C)

For intermediate-level Perl programmers, here are the changes required to implement the wrapper using Inline::C:

# Bit::Set using Inline::C
package Bit::Set;

use strict;
use warnings;
use Inline C => Config =>
    BUILD_NOISY => 1,
    CLEAN_AFTER_BUILD => 0,
    LIBS => '-L/media/chrisarg/Software-Dev/Bit/bit/build -lbit',
    INC => '-I/media/chrisarg/Software-Dev/Bit/bit/include';

use Inline C => <<'END_C';
#include "bit.h"

typedef struct Bit_T* Bit_T;

// Wrapper functions
Bit_T create_bitset(int length) {
    return Bit_new(length);
}

void free_bitset(Bit_T set) {
    Bit_free(&set);
}

int get_length(Bit_T set) {
    return Bit_length(set);
}

int get_count(Bit_T set) {
    return Bit_count(set);
}

void set_bit(Bit_T set, int index) {
    Bit_bset(set, index);
}

void clear_bit(Bit_T set, int index) {
    Bit_bclear(set, index);
}

int get_bit(Bit_T set, int index) {
    return Bit_get(set, index);
}

// Add more wrapper functions as needed
END_C

# Perl OO interface
sub new {
    my ($class, $length) = @_;
    my $handle = create_bitset($length);
    return bless { _handle => $handle }, $class;
}

# Add more methods similar to the FFI version

Modifications for Advanced Level (XS)

For advanced Perl programmers, here are the key changes required to implement using XS:

# Bit::Set.xs
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "bit.h"

typedef struct Bit_T* Bit_T;
typedef struct Bit_T_DB* Bit_T_DB;

MODULE = Bit::Set    PACKAGE = Bit::Set

PROTOTYPES: DISABLE

Bit_T
Bit_new(length)
    int length

void
Bit_free(set)
    Bit_T set
    CODE:
        Bit_free(&set);

int
Bit_length(set)
    Bit_T set

int
Bit_count(set)
    Bit_T set

void
Bit_bset(set, index)
    Bit_T set
    int index

# Add more functions as needed

And update the Makefile.PL:

use ExtUtils::MakeMaker;

WriteMakefile(
    NAME              => 'Bit::Set',
    VERSION_FROM      => 'lib/Bit/Set.pm',
    LIBS              => ['-L/media/chrisarg/Software-Dev/Bit/bit/build -lbit'],
    INC               => '-I/media/chrisarg/Software-Dev/Bit/bit/include',
    OBJECT            => '$(O_FILES)',
);

Conclusion

For most users, especially beginners, the FFI::Platypus approach offers the best balance of simplicity,
performance, and maintainability. It works well with your existing dynamic library build process and
provides a clean, object-oriented interface to the Bit_T and Bit_T_DB types. The implementation provided
covers all the core functionality while being extensible for additional features as needed.

My critique of Claude's choices

In this first post, we will not consider the implementation(s) which will be covered in separate posts, but will restrict ourselves to Claude's output with respect to the other questions. I give Claude bonus points for clearly identifying the niches for each potential choice:

1. Highighting that SWIG can support wrapping for other languages)
2. Proposing Foreign Function Interface for linking to the dynamic library for beginners.
3. A experienced programmer would opt for XS is not a surprise at all.

However:

• the choice of Inline for the intermediate user is head-scratching: it seems that the chatbot closed on the intermediate level of programming experience in the prompt, and the selection of the approach was driven entirely by the fact that the user could (presumably) do more stuff in C.
• SWIG was not considered as suitable (perhaps because few people in the training databases use SWIG) for implementing at any level. Without going into the specifics of the implementation though, I'd feel comfortable opting for FFI as an initial step for largely the reasons identified by Claude. We will have more things to say about the FFI implementation in the subsequent post in this series.

Note, I did not use the word understanding, as I do not think that LLMs can understant: they are merely noisy statistical pattern generators that can be tasked to create rough solutions for refining.
I alerted the bot to the (substantial) risk of hallucinations and decreased

Originally published at Perl Weekly 727

Hi there!

Recently I ran a poll asking Which versions of Perl do you use at work?. I asked it both in the The Perl Community group on Facebook and in the Perl Maven community Telegram Channel and the Perl 5 Telegram channel.

There were 75 votes in the Facebook group: 1% use 5.42; 17% use 5.40; 30% use 5.38; 5% use 5.36; 29% use 5.22-5.34; 13% use 5.12-5.20; 5% use 5.10 or older.

There were 29 votes in the Telegram group(s): 14% use 5.42; 31% use 5.40; 34% use 5.38; 7% use 5.36; 38% use 5.22-5.34; 17% use 5.12-5.20; 7% use 5.10 or older. Yes, people could select multiple answers.

You can still go an vote in either of those polls.

Many people commented that they use the version of perl that comes with the OS, to which Dave Cross posed Stop using your system Perl. I don't fully agree, but he has a point.

I don't recall what exactly prompted me to do this, but a few days ago I started to write a book about OOP - Object Oriented Perl. I took some of the slides I had for my advanced Perl course, started to update them and started to write explanations around the examples. As I write them I post the articles in the Perl Community group on Facebook and on my LinkedIn page. If you use Facebook I'd recommend you join that group and if you use LinkedIn I'd recommend you follow me. If you would like to go further and connect with me on LinkedIn, please include a message saying that you are a reader of the Perl Weekly newsletter so I'll have some background.

Besides publishing them on the social sites I also collect the writings about Perl OOP on my web site and I also started to publish the book on Leanpub.

As I am sure you know editing the Perl weekly, writing these examples and articles and the book takes a lot of time that I should spend earning money by helping my clients with their Perl code-base. Luckily there are some people who support me financially via Patreon, GitHub. (Hint: it would be awesome if you'd also sponsor me with $10/month.)

There are many people who like to get something 'tangible' to really justify their support. If you are such a person I created the Leanpub book especially for you. You can buy the Perl OOP book and some of my other books getting a pdf and an epub (for Kindle) version. You will both support my work and get an e-book. You will also get all the new versions of the book as I update it.

Enjoy your week!

--
Your editor: Gabor Szabo.

Articles

Stop using your system Perl

For years Dave has been telling us to use our own version of Perl and not to rely on the one that comes with our version of the Operating System. There is a lot of merit in what he is say and what he is writing in this article. I personally would go to the most extreme and use a Docker container for better separation and to make the development, testing, and production environments as similar as possible. With that said I am not 100% sold on the idea. I do understand the value in using the perl that comes with the OS and the perl modules that can be installed with the package-management system of the operating system. (e.g. apt/yum). See an extensive discussion on the topic.

Analysing FIT data with Perl: producing PNG plots

FIT files record the activities of people using devices such as sports watches and bike head units.

Lexical Method in Perl v5.42

There are all kinds of new and nice ways to write Perl code. Reading this article you'll learn about the new, experimental 'class' feature a bit more.

AWS DynamoDB

AWS DynamoDB is a fully managed NoSQL database service provided by AWS.

Discussion

Is there a (standardized) way to declare dependencies to a directory in a cpanfile?

How do you manage the dependencies of a Perl module that depends on another perl module (distribution) developed in the same monorepo?

Is there a tool that solves the constraint problem for Perl packages?

What happens when the same module is required both by your code and by one of your dependencies? Which version of that shared dependency will be installed?

Jobs

Where Are All the Perl Jobs in 2025?

Just a few days ago I asked on our Telegram channel which companies use Perl for application development and now I see this post. Apparently there is a new web-site listing 'Perl jobs'. I don't know who is behind that site, but the background image has 'use strict' and 'use warnings' so it isn't that bad.

How to find Perl job in 2025?

A discussion with some good (and some bad) suggestions there.

Full Stack Developer with strong Perl expertise

Perl Developer

Perl

This week in PSC (196) | 2025-06-19

The Weekly Challenge

The Weekly Challenge by Mohammad Sajid Anwar will help you step out of your comfort-zone. You can even win prize money of $50 by participating in the weekly challenge. We pick one champion at the end of the month from among all of the contributors during the month, thanks to the sponsor Lance Wicks.

The Weekly Challenge - 328

Welcome to a new week with a couple of fun tasks "Replace all ?" and "Good String". If you are new to the weekly challenge then why not join us and have fun every week. For more information, please read the FAQ.

RECAP - The Weekly Challenge - 327

Enjoy a quick recap of last week's contributions by Team PWC dealing with the "Missing Integers" and "MAD" tasks in Perl and Raku. You will find plenty of solutions to keep you busy.

Missing Integers Don’t Make Me MAD, Just Disappointed

Taking advantage of Perl's hash lookup speed, O(1) time complexity. Keeps the implementation readable and concise.

TWC327

A compact, correct, and Perl-savvy solution sets. The post’s minimalist style reflects a strong grasp of both Perl idioms and the challenge requirements.

Missing Mad

A well-structured, educational post that highlights the expressive power of Raku while staying true to challenge constraints. The inclusion of verbose output, clean modularity, and idiomatic constructs makes it an excellent read for both Raku learners and seasoned scripters.

Perl Weekly Challenge: Week 327

Offers elegant solutions in Raku and also provides working Perl equivalents. It balances code with commentary.

Absolutely Missing Differences

A technically sharp, creative post—especially with the use of PDL.

Perl Weekly Challenge 327

Practical, ready-to-run code with clear explanation and includes both straightforward Perl and more advanced PDL solutions.

Missing and Mad

It provides concise and elegant solutions to both challenges with clear explanations of the reasoning behind them. The use of idiomatic Perl (e.g., map, grep, keys, and smart use of hashes) is idiomatic and effective.

Missing Absolute Distance

It demonstrates good teaching style by explaining the problem, providing example inputs/outputs, and showing step-by-step approaches. The inclusion of multiple languages (Raku, Perl, Python, Elixir) is very valuable for readers interested in cross-language algorithm implementations.

Missing and mad

Both solutions are clean, efficient, and well-documented. It prioritises readability over one-liners—a good choice for maintainability.

The Weekly Challenge #327

Using none to check if a number is missing is idiomatic and easy to read. Setting $"=', ' for array printing is a good touch.

I Miss My Mind the Most

Away week, still got Rust for us. As always, a detailed and descriptive approach makes the reading fun.

The weekly challenge 327

This post is a well-written, thoughtful exploration of two classic array problems, each solved elegantly in both Python and Perl, with a particular focus on bitarray usage and iteration techniques.

Perl Tutorial

A section for newbies and for people who need some refreshing of their Perl knowledge. If you have questions or suggestions about the articles, let me know and I'll try to make the necessary changes. The included articles are from the Perl Maven Tutorial and are part of the Perl Maven eBook.

Videos

GPW 2025 - Mark Overmeer - Mid-life upgrade for MailBox

Weekly collections

NICEPERL's lists

Great CPAN modules released last week.

Events

Purdue Perl Mongers - TBA

July 9, 2025

You joined the Perl Weekly to get weekly e-mails about the Perl programming language and related topics.

Want to see more? See the archives of all the issues.

Not yet subscribed to the newsletter? Join us free of charge!

(C) Copyright Gabor Szabo
The articles are copyright the respective authors.

Q: What is the State of the Art for creating Artificial Neural Networks with Perl?

Why would I want to use an ANN in the first place? Well, maybe I have some crime/unusual incident data that I want to correlate with the Phases of the Moon to test the Lunar Effect, but the data is noisy, the effect is non-linear or confounded by weather. For whatever reason you want to “learn” a general pattern going from input to output, neural networks are one more method in your data science toolbox.

A search of CPAN for Neural Networks yields one page of results for you to sift through. The back propagation algorithm is a nice exercise in programming and it attracted a few attempts at the beginning of the century, starting with Statistics::LTU in 1997 before there was an AI namespace in CPAN. Neural networks then get their own namespace, leading to AI::NeuralNet::BackProp, AI::NeuralNet::Mesh, AI::NeuralNet::Simple (for those wanting a gentle introduction to AI). Perl isn’t one for naming rigidity, so there’s also AI::Perceptron, AI::NNFlex, AI::NNEasy and AI::Nerl::Network (love the speeling). AI::LibNeural is the first module in this list to wrap an external C++ library for use with Perl.

Most of these have been given the thumbs up (look for ++ icons near the name) by interested Perl users to indicate that it’s been of some use to them. It means the documentation is there, it installs and works for them. Is it right for you? NeilB puts a lot of work into his reviews, but hasn’t scratched the AI itch yet, so I’ll have to give one a try.

Sometimes trawling the CPAN dredges up interesting results you weren’t thinking about. I had no idea we had AI::PSO for running Particle Swarm Optimizations, AI::DecisionTree or AI::Categorizer to help with categorization tasks and AI::PredictionClient for TensorFlow Serving. Maybe I’ll come back to these one day. Searching specifically for [Py]Torch gets you almost nothing, but I did find AI::TensorFlow::Libtensorflow which provides bindings for the libtensorflow deep learning library.

MXNet

A flexible and efficient library for Deep Learning

AI::MXNet gets lots of love from users (not surprising given the popularity of convolutional neural networks). With a recent update for recurrent neural networks (RNN) in June 2023 and the weight of an Apache project behind the underlying library, it should be the obvious choice. But checking out the project page and decision-making disaster strikes!

MXNet had a lot of work on it, but then was retired in Sep 2023 because the Project Management Committee were unresponsive over several months, having uploaded their consciousnesses to a datacube in Iceland or maybe they just went on to other things because of … reasons.

It should still be perfectly fine to use. That Apache project had 87 contributors, so I expect it to be feature-rich and generally bug-free. Any bugs in the Perl module could be reported/fixed and you always have the source code for the library to hack on to suit your needs. I’ll skip it this time because I’m really only after a simple ANN, not the whole Deep Learning ecosystem, and I couldn’t find the package in the Fedora repository (adding the extra friction of building it myself).

FANN

A Fast Artificial Neural Network

FANN has been around for over 15 years is generally faster to train and run than either TensorFlow or PyTorch. The speed and lightweight nature make it ideal for embedded systems. Its smaller community may have an impact on your choice. From my 10 minute inspection, AI::FANN seemed to be the easier to get up to speed with. It had a short, simple example at the top of the docs that I could understand and run without much fuss.

In contrast, AI::MXNet leads with a Convolutional Neural Net (CNN) for recognizing hand-written digits in the MNIST dataset. It gives you a feel for the depth of the feature set, at the risk of intimidating the casual reader. Mind you, if I was looking for image classification (where CNNs shine) or treating history as an input (using RNNs as mentioned above), I’d put the time in going through AI::MXNet.

The downside to the original FANN site is the documentation consists of a series of blog posts that tell you all the things you can do, but not how to do them. You’re best bet is to read the examples source code like all the other C programmers out there.

Getting Started

Installation was easy. You just need the FANN build libraries (header files, etc) and the Perl module that interfaces to them. You could build from source or get libfan-dev on Ubuntu. For me on Fedora, it was just a matter of

dnf install fann-devel
cpanm AI::FANN

(See Tools for using cpanm)

To get started, I tried out the XOR example in the docs. XOR is a classic example of how a multi-layered perceptron (MLP) can tackle problems that are not linearly separable. The hidden layers of the MLP can solve problems inaccessible to single layer perceptrons. It gave me confidence in using a data structure to initialize the network and importing data from a file. An hour later, I was already scratching the itch that drew me to neural networks in the first place.

Network design and evaluation

A nice introduction is FANN’s step-by-step guide which will take you through a bit about learning rates and activation functions as you consider how to build and tweak your first neural network. There are few heuristics to go by, so just start playing around until you get a result.

Be careful that too many neurons in the hidden layers will lead to overfitting of your data. You’ll end up with a network that can reproduce the training data perfectly, but fail to learn the underlying signal you wanted to discover. You might start with something between the number of input and output neurons. And be aware that machine learning algorithms are data-hungry.

Activation functions can affect how long it takes to train your network. Previous experience with other neural network tools way back in 2005 taught us the importance of normalizing the input, ideally to a range of [-1, 1], because most of the training time was spent just adjusting the weights to the point where the real learning could begin. Use your own judgement.

While we see the train_on_data and run methods in the example, you have to look down in the docs for the test method which you’ll need to evaluate the trained network. The MSE method will tell you the Mean Squared Error for your model and lower values are better. There’s no documentation for it yet, but it should do what it says on the tin.

A network that gives you rubbish is no good, so we need to evaluate how well it has learned on the training data. The usual process is to split the dataset into training and testing sets, reserving 20-30% of the data for testing. Once the network has finished training, its weights are fixed and then run on the testing data with the network’s output compared with the expected output given in the dataset.

Cross-validation is another popular method of evaluation, splitting the dataset into 10 subsets where you train on 9 sets and test on the 10th, rotating the sets to improve the network’s response. Once you are satisfied with the performance of your network, you are ready to run it on live data. Just remember to sanity check the results while you build trust in the responses.

Going back every time and manually creating networks with different sizes of layers sounds tedious. Ideally, I’d have a script that takes the network layers and sizes as arguments and returns the evaluation score. Couple this with the Minion job queue from Mojolicious (it’s nice!) and you’d have a great tool for finding the best available neural network for the given data while you’re doing other things.

The Missing Datafile Format

The one thing not easy to find on the website is the file format specification for the datafiles, so this is what I worked out. They are space separated files of integers or floats like this

Number_of_runs Number_of_inputs Number_of_outputs
Input row 1
Output row 1
Input row 2
Output row 2
...

This is a script that will turn an array of arrayrefs from the XOR example into the file format used by libfann.

use v5.24; # postfix dereferencing is cool

my @xor_data = ( [[-1, -1], [-1] ],
                 [[-1, 1], [1] ],
                 [[1, -1], [1] ],
                 [[1, 1], [-1] ] ); 
write_datafile('xor.data', @xor_data);

sub write_datafile {
    my ($filename, @data) = @_;

    open my $fh, '>', $filename;
    my ($in, $out) = $data[0]->@*;
    say $fh join q{ }, scalar @data, scalar @$in, scalar @$out; 

    for my $test (@data) {
        say $fh join q{ }, $test->[0]->@*;
        say $fh join q{ }, $test->[1]->@*;
    }
    close $fh;
}

Your turn ...

Have you used any of these modules? Share your experience to help the next person choose. Have I missed anything or got something wrong? Let us know in the comments below.

Thank you for your time!

Image credit: “Perceptron” by fdecomite is licensed under CC BY 2.0

Recently, Gabor ran a poll in a Perl Facebook community asking which version of Perl people used in their production systems. The results were eye-opening—and not in a good way. A surprisingly large number of developers replied with something along the lines of “whatever version is included with my OS.”

If that’s you, this post is for you. I don’t say that to shame or scold—many of us started out this way. But if you’re serious about writing and running Perl in 2025, it’s time to stop relying on the system Perl.

Let’s unpack why.

What is “System Perl”?

When we talk about the system Perl, we mean the version of Perl that comes pre-installed with your operating system—be it a Linux distro like Debian or CentOS, or even macOS. This is the version used by the OS itself for various internal tasks and scripts. It’s typically located in /usr/bin/perl and tied closely to system packages.

It’s tempting to just use what’s already there. But that decision brings a lot of hidden baggage—and some very real risks.

Which versions of Perl are officially supported?

The Perl Core Support Policy states that only the two most recent stable release series of Perl are supported by the Perl development team [Update: fixed text in previous sentence]. As of mid-2025, that means:

• Perl 5.40 (released May 2024)

• Perl 5.38 (released July 2023)

If you’re using anything older—like 5.36, 5.32, or 5.16—you’re outside the officially supported window. That means no guaranteed bug fixes, security patches, or compatibility updates from core CPAN tools like ExtUtils::MakeMaker, Module::Build, or Test::More.

Using an old system Perl often means you’re several versions behind , and no one upstream is responsible for keeping that working anymore.

Why using System Perl is a problem

1. It’s often outdated

System Perl is frozen in time—usually the version that was current when the OS release cycle began. Depending on your distro, that could mean Perl 5.10, 5.16, or 5.26—versions that are years behind the latest stable Perl (currently 5.40).

This means you’re missing out on:

• New language features (builtin, class/method/field, signatures, try/catch)

• Performance improvements

• Bug fixes

• Critical security patches

• Support: anything older than Perl 5.38 is no longer officially maintained by the core Perl team

If you’ve ever looked at modern Perl documentation and found your code mysteriously breaking, chances are your system Perl is too old.

2. It’s not yours to mess with

System Perl isn’t just a convenience—it’s a dependency. Your operating system relies on it for package management, system maintenance tasks, and assorted glue scripts. If you install or upgrade CPAN modules into the system Perl (especially with cpan or cpanm as root), you run the risk of breaking something your OS depends on.

It’s a kind of dependency hell that’s completely avoidable— if you stop using system Perl.

3. It’s a barrier to portability and reproducibility

When you use system Perl, your environment is essentially defined by your distro. That’s fine until you want to:

• Move your application to another system

• Run CI tests on a different platform

• Upgrade your OS

• Onboard a new developer

You lose the ability to create predictable, portable environments. That’s not a luxury— it’s a requirement for sane development in modern software teams.

What you should be doing instead

1. Use perlbrew or plenv

These tools let you install multiple versions of Perl in your home directory and switch between them easily. Want to test your code on Perl 5.32 and 5.40? perlbrew makes it a breeze.

You get:

• A clean separation from system Perl

• The freedom to upgrade or downgrade at will

• Zero risk of breaking your OS

It takes minutes to set up and pays for itself tenfold in flexibility.

2. Use local::lib or Carton

Managing CPAN dependencies globally is a recipe for pain. Instead, use:

• local::lib: keeps modules in your home directory.

• Carton: locks your CPAN dependencies (like npm or pip) so deployments are repeatable.

Your production system should run with exactly the same modules and versions as your dev environment. Carton helps you achieve that.

3. Consider Docker

If you’re building larger apps or APIs, containerising your Perl environment ensures true consistency across dev, test, and production. You can even start from a system Perl inside the container—as long as it’s isolated and under your control.

You never want to be the person debugging a bug that only happens on production, because prod is using the distro’s ancient Perl and no one can remember which CPAN modules got installed by hand.

The benefits of managing your own Perl

Once you step away from the system Perl, you gain:

• Access to the full language. Use the latest features without backports or compatibility hacks.

• Freedom from fear. Install CPAN modules freely without the risk of breaking your OS.

• Portability. Move projects between machines or teams with minimal friction.

• Better testing. Easily test your code across multiple Perl versions.

• Security. Stay up to date with patches and fixes on your schedule, not the distro’s.

• Modern practices. Align your Perl workflow with the kinds of practices standard in other languages (think virtualenv, rbenv, nvm, etc.).

“But it just works…”

I know the argument. You’ve got a handful of scripts, or maybe a cron job or two, and they seem fine. Why bother with all this?

Because “it just works” only holds true until:

• You upgrade your OS and Perl changes under you.

• A script stops working and you don’t know why.

• You want to install a module and suddenly apt is yelling at you about conflicts.

• You realise the module you need requires Perl 5.34, but your system has 5.16.

Don’t wait for it to break. Get ahead of it.

The first step

You don’t have to refactor your entire setup overnight. But you can do this:

• Install perlbrew and try it out.

• Start a new project with Carton to lock dependencies.

• Choose a current version of Perl and commit to using it moving forward.

Once you’ve seen how smooth things can be with a clean, controlled Perl environment, you won’t want to go back.

TL;DR

Your system Perl is for your operating system—not for your apps. Treat it as off-limits. Modern Perl deserves modern tools, and so do you.

Take the first step. Your future self (and probably your ops team) will thank you.

The post Stop using your system Perl first appeared on Perl Hacks.