PAGI 0.001017 is on CPAN. The headline feature is HTTP/2 support in PAGI::Server, built on the nghttp2 C library and validated against h2spec's conformance suite. HTTP/2 is marked experimental in this release -- the protocol works, the compliance numbers are solid, and we want production feedback before dropping that label.

This post focuses on why h2c (cleartext HTTP/2) between your reverse proxy and backend matters, and how PAGI::Server implements it.

The Quick Version

# Install the HTTP/2 dependency
cpanm Net::HTTP2::nghttp2

# Run with HTTP/2 over TLS
pagi-server --http2 --ssl-cert cert.pem --ssl-key key.pem app.pl

# Run with cleartext HTTP/2 (h2c) -- behind a proxy
pagi-server --http2 app.pl

Your app code doesn't change. HTTP/2 is a transport concern handled entirely by the server. The same async handlers that serve HTTP/1.1 serve HTTP/2 without modification.

"I Have nginx in Front, Why Do I Care?"

If nginx terminates TLS and speaks HTTP/2 to clients, why does the backend need HTTP/2 too?

The answer is h2c -- cleartext HTTP/2 between your proxy and your application server. No TLS overhead, but all of HTTP/2's protocol benefits on the internal hop: stream multiplexing over a single TCP connection, HPACK header compression (especially effective for repetitive internal headers like auth tokens and tracing IDs), and per-stream flow control so a slow response on one stream doesn't block others.

The practical wins: fewer TCP connections between proxy and backend (one multiplexed h2c connection replaces a pool of HTTP/1.1 connections), less file descriptor and kernel memory pressure, and no TIME_WAIT churn from connection recycling.

Where h2c Matters

gRPC requires HTTP/2 -- it doesn't work over HTTP/1.1 at all. If you're building gRPC services, h2c is mandatory.

API gateway fan-out is where multiplexing shines. When your gateway fans out to 10 backend services per request, h2c means 1-2 connections per backend instead of a pool of 50-100.

Service mesh environments (Envoy/Istio sidecars) default to HTTP/2 between services. A backend that speaks h2c natively means one less protocol translation.

A Note on Proxies

Not all proxies handle h2c equally:

• Envoy has the best h2c upstream support with full multiplexing
• Caddy makes it trivial: reverse_proxy h2c://localhost:8080
• nginx supports h2c via grpc_pass for gRPC workloads, but its generic proxy_pass doesn't support proxy_http_version 2.0

For full multiplexing to backends, Envoy or Caddy are better choices than nginx today.

HTTP/2 Over TLS -- No Proxy Required

h2c isn't the only mode. PAGI::Server also does full HTTP/2 over TLS with ALPN negotiation:

pagi-server --http2 --ssl-cert cert.pem --ssl-key key.pem app.pl

This is useful when you don't want the overhead or complexity of a reverse proxy -- internal tools, admin dashboards, development servers, or any app where the traffic doesn't justify a separate proxy layer. Browsers get HTTP/2 directly, with TLS, no nginx required.

What PAGI::Server Does

Dual-Mode Protocol Detection

With TLS, PAGI::Server uses ALPN negotiation during the handshake -- advertising h2 and http/1.1, letting the client choose. The protocol is decided before the first byte of application data.

Without TLS (h2c mode), PAGI::Server inspects the first 24 bytes of each connection for the HTTP/2 client connection preface. If it matches, the connection upgrades to HTTP/2. If not, it falls through to HTTP/1.1. Both protocols coexist on the same port, same worker -- no configuration needed beyond --http2.

Either way, HTTP/1.1 clients are still served normally. The server handles both protocols on the same port.

WebSocket over HTTP/2 (RFC 8441)

Most HTTP/2 implementations skip this. PAGI::Server supports the Extended CONNECT protocol from RFC 8441, which tunnels WebSocket connections over HTTP/2 streams. Multiple WebSocket connections multiplex over a single TCP connection instead of requiring one TCP connection each.

Compliance

Built on nghttp2 (the same C library behind curl, Firefox, and Apache's mod_http2). PAGI::Server passes 137 of 146 h2spec conformance tests (93.8%). The 9 remaining failures are in nghttp2 itself and shared with every server that uses it. Load tested with h2load at 60,000 requests across 50 concurrent connections with no data loss or protocol violations.

Full test-by-test results are published: HTTP/2 Compliance Results.

Multi-Worker and Tunable

HTTP/2 works in multi-worker prefork mode. Each worker independently handles HTTP/2 sessions:

pagi-server --http2 --workers 4 app.pl

Protocol settings are exposed for environments that need fine-tuning:

my $server = PAGI::Server->new(
    app   => $app,
    http2 => 1,
    h2_max_concurrent_streams => 50,   # default: 100
    h2_initial_window_size => 131072,  # default: 65535
    h2_max_frame_size => 32768,        # default: 16384
    h2_max_header_list_size => 32768,  # default: 65536
);

Most deployments won't need to touch these. The defaults follow the RFC recommendations.

Context in the Perl Ecosystem

Perl has had HTTP/2 libraries on CPAN (Protocol::HTTP2, Net::HTTP2), but application servers haven't integrated them with validated compliance testing. PAGI::Server is the first to publish h2spec results and ship h2c with automatic protocol detection alongside HTTP/1.1. If you're currently running Starman, Twiggy, or Hypnotoad, none of them offer HTTP/2.

What Else Is in 0.001017

The rest of the release is operational improvements:

• Worker heartbeat monitoring -- parent process detects workers with blocked event loops and replaces them via SIGKILL + respawn. Default 50s timeout. Only triggers on true event loop starvation; async handlers using await are unaffected.
• Custom access log format -- format strings with atoms like %a (address), %s (status), %D (duration).
• TLS performance fix -- shared SSL context via SSL_reuse_ctx eliminates per-connection CA bundle parsing. 26x throughput improvement at 8+ concurrent TLS connections.
• SSE wire format fix -- now handles CRLF, LF, and bare CR line endings per the SSE specification.
• Multi-worker fixes -- shutdown escalation, parameter pass-through, and various stability improvements.

Getting Started

# Install PAGI
cpanm PAGI

# Install HTTP/2 support (optional)
cpanm Net::HTTP2::nghttp2

# Run your app with HTTP/2
pagi-server --http2 app.pl

Links

• PAGI 0.001017 on CPAN
• HTTP/2 Compliance Results
• PAGI on GitHub
• h2spec -- HTTP/2 conformance testing tool

It's been a while since I commented on a Weekly Challenge solution, but here we are at week 360. Such a useful number. So divisible, so circular. It deserves twenty minutes.

Task 2: Word Sorter

The task

You are given a sentence. Write a script to order words in the given
sentence alphabetically but keep the words themselves unchanged.

# Example 1 Input: $str = "The quick brown fox"
#           Output: "brown fox quick The"
# Example 2 Input: $str = "Hello    World!   How   are you?"
#           Output: "are Hello How World! you?"
# Example 3 Input: $str = "Hello"
#           Output: "Hello"
# Example 4 Input: $str = "Hello, World! How are you?"
#           Output: "are Hello, How World! you?"
# Example 5 Input: $str = "I have 2 apples and 3 bananas!"
#           Output: "2 3 and apples bananas! have I"

The thoughts

This should be quite simple: split the words, sort them, put them back together. The sort should be case-insensitive.

join " ", sort { lc($a) cmp lc($b) } split(" ", $str);

Creeping doubt #1

Is converting to lowercase with lc the right way to do case-insenstive compares? Not really. Perl has the fc -- fold-case -- function to take care of subtleties in Unicode. We won't see those in simple ASCII text, but for the full rabbit hole, start with the documentation of fc.

Creeping doubt #2

Doing the case conversion inside the sort means that we will invoke that every time there's a string comparison, which will be quite redundant. We could (probably?) speed it up by pre-calculating the conversions once.

The solution

sub sorter($str)
{
    return join " ",
        map { $_->[0] }
        sort { $a->[1] cmp $b->[1] }
        map { [$_, fc($_)] }
        split(" ", $str);
}

This solution uses the idiom of Schwartzian transformation. Every word gets turned into a pair of [original_word, case_folded_word]. That array of pairs gets sorted, and then we select the original words out of the sorted pairs. This is best read bottom-up.

• split(" ", $str) -- turns the string into an array of words, where words are loosely defined by being white-space separated.
• map { [$_, fc($_)] } -- every word turns into a pair: the original, and its case-folded variation. The result is a list of array references.
• sort { $a->[1] cmp $b->[1] } -- sort by the case-folded versions. The result is still a list of array references.
• map { $_->[0] } -- select the original word from each pair
• join " " -- Return a single string, where the words are separated by one space.

Does it blend?

A quick benchmark shows that it is indeed faster to pre-calculate the case folding. This example used a text string of about 60 words.

           Rate oneline  pre_lc
oneline 32258/s      --    -45%
pre_lc  58140/s     80%      --

My intuition says that when the strings are much shorter, the overhead of the transform might not offset the gains in the sort, but as is so often true, my intuition is crap. This is the result for a string of five words:

oneline  470588/s      --    -59%
pre_lc  1142857/s    143%      --

Beautiful Perl series

This post is part of the beautiful Perl features series.
See the introduction post for general explanations about the series.

BLOCK : sequence of statements

Today's topic is the construct named BLOCK in the perl documentation: a sequence of statements enclosed in curly brackets {}. Both the concept and the syntax are common to many programming languages - they can also be found in languages of C heritage like Java, JavaScript and C++, among others; but Perl is different on various aspects - read on to dig into the details.

BLOCKs in Perl may be used:

1. as part of a compound statement, after an initial control flow construct like if, while, foreach, etc.;
2. as the body of a sub declaration (subroutine or function);
3. at any location where a single statement is expected. Obviously it would also be possible to just insert a plain sequence of statements, but enclosing them in a BLOCK has the advantage of creating a new delimited lexical scope so that the effect of inner declarations is guaranteed to end when control flow exits the BLOCK. Details about lexical scopes are discussed below;
4. as part of a do expression, so that the whole BLOCK becomes a value that can be inserted within a more complex expression. This may be convenient for clarity of thought in some algorithms, and also for avoiding a subroutine call when efficiency is at stake.

All modern programmming languages have constructs equivalent to usages 1 and 2, because these are crucial for structuring algorithms and for handling complexity in programming. Usage 3 is less common, and usage 4 is quite particular to Perl. The next chapters will cover these various aspects in more depth.

Lexical scope

In all usage situations listed above, a Perl BLOCK always opens a new lexical scope, i.e. a portion of code that delimits the effect of inner declarations. Things that can be temporarily declared inside a lexical scope are:

• lexical variables, temporarily binding a variable name to a memory location on the stack - declared through keywords my or state;
• lexical pragmata, temporarily importing semantics into the current BLOCK - introduced through keywords use or no;
• (beginning with Perl 5.18) lexical subroutines, only accessible within the scope - also declared through keywords my or state.

When a BLOCK is used as part of a compound statement (if, foreach, etc.), the initial clause before the BLOCK is already part of the lexical scope, so that variables declared in that clause can then be used within the BLOCK:

foreach my $member (@list) {
  work_with($member); # here $member can be used
}
say $member;          # ERROR : $member is no longer in scope

This is also true when a compound statement has several clauses and therefore several BLOCKs, like if (...) {...} elsif {...} else {...}. Further examples, together with detailed explanations, can be found in perlsub.

Declarations of variables, pragmata or subroutines have the following facts in common:

• they take effect starting from the statement after the declaration. Technically declarations can occur anywhere in the BLOCK; the most common usage is to put them at the beginning, but there is no obligation to do so;
• they may temporarily shadow the effect of declarations in higher scopes;
• their effect ends when control flow exits the BLOCK, whatever the exit cause may be (normal end of the block, explicit exit through instructions like return, next or goto, or exit because an exception was encountered).

Declarations in lexical scopes have effects both at compile-time (the Perl interpreter temporarily alters its parsing rules) and at runtime (the interpreter temporarily allocates or releases resources). For example in the following snippet:

{
  my $db_handle   = DBI->connect(@db_connection_args);
  my @large_array = $db_handle->selectall_array($some_sql, @some_bind_values);

  open my $file_handle, ">", $output_file or die "could not open $output_file: $!";
  print $file_handle formatted_ouput($_) foreach @large_array;
} 

the interpreter knows at compile-time that variables $db_handle, @large_array and $file_handle are allowed within the BLOCK, but not outside of it, so it can check statically for typos or other misuses of variables names; then at runtime the interpreter will dynamically allocate and release memory and external handles when control flow crosses the BLOCK. This is quite similar to what happens in a statically typed language like Java. By contrast, Python, often grouped in the same family as Perl because it is also a dynamically-typed language, does not have the same treatment of lexical scopes.

Lexical scopes in Python are not like Perl

In Python, there is no generally available construct as versatile as a Perl BLOCK. Subsequences of statements are expressed through indentation, but this is only allowed as part of a function definition or as part of a compound statement. A compound statement must always start with a keyword (if, for, while, etc.) that opens the header clause and is followed by a suite:

if is_success():
    summary = summarize_results()
    report_to_user(summary)
    cleanup_resources()

A 'suite' in Python is not to be confused with a 'block'. The official documentation is very careful about the distinction, but many informal texts in Python literature make the confusion; yet the difference is quite important because:

• a Python block, "a piece of Python program text that is executed as a unit", only occurs within a module, a function body or a class definition (https://docs.python.org/3/reference/executionmodel.html#structure-of-a-program);
• a Python suite, "a group of statements controlled by a clause", occurs whenever a clause in a compound statement expects to be followed by some instructions.

Blocks and suites look similar, because they are both expressed as indented sequences of statements; but the difference is that a 'block' opens a new lexical scope, while a 'suite' does not. In particular, this means that variables declared in a compound statement are still available after the statement has ended, which is quite surprising for programmers coming from a C-like culture (including Perl and Java). So for example

for i in [1, 2, 3]:
    pass
print(i)

is valid Python code and prints 3. This example is taken from https://eli.thegreenplace.net/2015/the-scope-of-index-variables-in-pythons-for-loops/ which does a very good job at explaining why Python in this respect works differently from many other languages.

Lexical scope vs dynamic scope

In addition to traditional lexical scoping, Perl also has another construct named dynamic scoping, introduced through the keyword local. Dynamic scoping is a vestige from Perl1, but still very useful in some specific use cases; it will be discussed in a future post in this series. For the moment let us just say that in all common situations, lexical scoping is the most appropriate mechanism for working with variables guaranteed not to interfere with the global state of the program.

Lexical variables

Lexical variables in Perl are introduced with the my keyword. Several variables, possibly of different types, can be declared and initialized in one single statement:

my @table               = ([qw/x y z/], [1, 2, 3], [9, 8, 7]);
my ($nb_rows, $nb_cols) = (scalar @table, scalar $table[0]->@*); 
my (@db_connection_args, %select_args);

Variable initialization and list destructuring

Lexical variables can only be used starting from the statement after the declaration. Therefore it is illegal in Perl to write something like:

my ($x, $y, $z) = (123, $x + 1, $x + 2);

because at the point where expression $x + 1 is encountered, variable $x is not yet in scope. By contrast, Java would accept int x = 123, y = x + 1, z = x + 2, or JavaScript would accept let x = 123, y = x + 1, z = x + 2, because in those languages variable initializations occur in sequence, while Perl starts by evaluating the complete list on the right-hand side of the assignment, and then distributes the values into variables on the left-hand side.

Python is like Perl: it does not accept x, y = 123, x + 1 because x is not defined in the right-hand side. Both Perl and Python had from the start the notion of "destructuring" a list into several individual variables. Other languages adopted similar features much later:

• JavaScript has an advanced mechanism of destructuring since ES6 (2015), that can be applied not only to lists, but also to objects (records). For destructuring a list the syntax variables must be put in angle brackets on the left-hand side of the assignment, in order to avoid ambiguity with sequences of ordinary assignments:

let x = 123, y = x + 1, z = x + 2;  // sequence of assignments
let [a, b, c] = [123, 124, 125];    // list destructuring

• The Amber project for Java recently introduced several mechanisms for pattern matching, which is quite close to the idea of destructuring. However for the moment it can only be used for destructuring records, but not yet for lists.

Coming back to Perl, list destructuring has always been part of common idioms, notably for:

• extracting items from command-line arguments

my ($user, $password, @others) = @ARGV;

• extracting items from the argument list to a subroutine

my ($height, $width, $depth) = @_;

• switching variables

($x, $y) = ($y, $x);

Shadowing variables from higher scopes

Like in other languages, a lexical variable in Perl can shadow another variable of the same name at a higher lexical scope. However, the shadowing effect only starts at the statement after the declaration of the variable. As a result, the shadowed value can still be used in the initializing expression:

my $x = 987;
{ my ($x, $y) = (123, $x + 1);
  say "inner scope, x is $x and y is $y"; # "inner scope, x is 123 and y is 988"
}
say "outer scope, x is $x";               # "outer scope, x is 987"

Now let us see how other dynamically typed languages handle variable shadowing.

Shadowing variables in Python

Python has no explicit variable declarations; instead, any assignment instruction implicitly declares the target of the assignment to be a lexical variable in the current lexical scope.

def foo():
    x = 123 # declares lexical variable x
    y = 456 # declares lexical variable y
    x = 789 # assigns a new value to existing variable x

Since the intention of declaring is not explicitly stated by the programmer, the interpreter is of little help for detecting errors that would be identified as typos in other languages. In the example above, one could suspect that the intent was to declare a z variable instead of assigning a new value to x.

If an assignment occurs in the middle of a lexical scope, the corresponding variable is nevertheless treated as being lexical from the very beginning of the scope. As a consequence, newcomers to Python can easily be surprised by an UnboundLocalError, which can be shortly demonstrated by this example from the official documentation:

x = 10
def foo():
    print(x)
    x += 1

foo()

Here the assignment x += 1 implicitly declares x to be a lexical variable for the whole body of the foo() function, even if the assignment comes at the end. In this situation the print() statement raises an exception because at this point lexical variable x is not bound to a value. By contrast, if the assignment is commented out

x = 10
def foo():
    print(x)
    # x += 1

foo()

the program happily prints 10, because here x is no longer interpreted as a lexical variable, but as the global x.

Python statements global and nonlocal can instruct the parser that some specific variables should not be declared in the current lexical scope, but should instead be taken from the global module scope, or, in case of nested functions or classes, from the next higher scope. So on this respect, Python programming is just the opposite to Perl or Java : instead of explicitly declaring lexical variables, one must explicitly declare the variables that are not lexical. Furthermore, since such declarations apply to the whole current lexical scope, independently of the place where they are inserted, it is an exclusive choice : any use of a named variable must be either from the current lexical scope or from a higher scope or from the global scope. Therefore it is not possible, like in Perl, to use the value of global x in the initialization expression for local lexical x.

Shadowing variables in JavaScript

The historical construct for declaring lexical variables in JavaScript was through the var keyword, which is still present in the language. The behaviour of var is quite similar to Python lexical variables: variables appear to exist even before they are declared (which is called hoisting in JavaScript); they are scoped by functions or modules, not by blocks, so they still hold values afer exiting from the block; and the interpreter does not complain if a variable is declared twice. For all these reasons, var is now deprecated, replaced since ES6 (2015) by keywords const (for variables that do not change after initialization) or let (for mutable variables).

These new constructs indeed introduced more security in usage of lexical variables in JavaScript: such variables can no longer be used after exiting from the block, and redeclarations raise syntax errors. Yet one ambiguity remains: the shadowing effect of a variable declared with let does not start at the location of the declaration, but at the beginning of the enclosing block. This is no longer called "hoisting", but still it means that from the beginning of the block that variable name shadows any variable with the same name in higher scopes. This is called temporal dead zone in JavaScript literature.

Shadowing is prohibited in Java

Java has no ambiguity with shadowing ... because it has a more radical approach: it raises an exception when a variable is declared in an inner block with a name already in use at a higher scope! The following snippet

public class ScopeDemo {
  public static void main(String[] args) {
    int x = 987;
    {
      int x = 123, y = x + 1, z = x + 2;
      System.out.println("here x is " + x + " and y is " + y);
    }
    System.out.println("here x is " + x);
  }
}

yields:

ScopeDemo.java:6: error: variable x is already defined in method main(String[])
      int x = 123, y = x + 1, z = x + 2;
          ^
1 error
error: compilation failed

Lexical pragmata

In Perl, lexical scopes are not only used to control the lifetime of lexical variables: they are also used for lexical pragmata that temporarily alter the behaviour of the interpreter, either by adding some semantics (through the keyword use) or by removing some semantics (through the keyword no). Here is an example of one very common idiom:

use strict;
use warnings;
foreach my $user (get_users_from_database()) {
  no warnings 'uninitialized';
  my $body = "Dear $user->{firstname} $user->{lastname}, bla bla bla";
  ...
}

At the beginning of the program, the warnings pragma is activated, because this is general good practice, so that the interpreter can detect suspect situations and warn about them. But when working with a $user record from the database, some fields might be undef, which is OK, there is no reason to issue a warning - so within that BLOCK the interpreter is instructed to treat undefined data as empty strings, without complaining.

In a similar vein, it is sometimes necessary to alleviate the controls performed by use strict, in particular on the subject of symbolic references. This control forbids programmatic insertion of new subroutines into the symbol table of a module, a safe measure of prevention; yet this feature is powerful and very useful in some specific situations - so when needed, one can temporarily disable the control:

foreach my $method_name (@list_of_names) {
  no strict 'refs';
  *{$method_name} = generate_closure_for($method_name);
}

This technique is used quite extensively for example in the Object-Relational Mapping module DBIx::DataModel for generating methods that implement navigation from one table to another related table. The source code can demonstrate usage patterns.

Some pragmata can also reinforce controls instead of alleviating them. One very good example is the autovivification module, which changes the default behaviour of Perl on implicit creation of intermediate references:

my $tree;                  # at this point, $tree is undef
$tree->{foo}{bar}[1] = 99; # no error; now $tree is {foo => {bar => [undef, 99]}}

Autovivification can be very handy, but it can be dangerous too. If we want to be on the safe side, we can write

{ no autovivification qw/fetch store/;
  my $tree;                  # at this point, $tree is undef
  $tree->{foo}{bar}[1] = 99; # ERROR: Can't vivify reference
}

Like for lexical variables, lexical pragmata can be nested, the innermost use or no declaration temporarily shadowing previous declarations on the same pragma.

Other examples of lexical pragmata include bigint, which transparently transforms all arithmetic operations to work with instances of Math::BigInt; or the incredible Regexp::Grammars module that adds grammatical parsing features to Perl regexes. The perlpragma documentation explains how module authors can implement new lexical pragmata.

'do': transform a BLOCK into an expression

In all examples seen so far, BLOCKs were treated as statements; but thanks to the do BLOCK construct, it is also possible to insert a BLOCK anywhere in an expression. The value from the last instruction in the BLOCK is then processed by operators in the expression. This is very convenient for performing a small computation in-place, either for clarity or for efficiency reasons.

The first example is a cheap version of XML entity encoding, slight adaptation from my module Excel::ValueWriter::XLSX:

my %ENTITY_TABLE = ( '<' => '&lt;', '>' => '&gt;', '&' => '&amp;' );
my $entity_regex = do {my $chars = join "", keys %ENTITY_TABLE; qr/[$chars]/};
...
$text =~ s/($entity_regex)/$ENTITY_TABLE{$1}/g; # encode entity characters in $text

The second example is from the cousin module Excel::ValueReader::XLSX. Here we are parsing the content of table in an Excel sheet, and the data is returned either in the form of a list of arrayrefs (plain values), or in the form of a list of hashrefs (column name => value), depending on an option given by the caller:

my $row = $args{want_records} ? do {my %r; @r{@{$args{columns}}} = @$vals; \%r}
                              : $vals;

If the caller wants records, the do block performs a hash slice assignment into a lexical hash variable to create a new record on the fly.

Wrapping up

Thanks to BLOCKs, lexical scoping can be introduced very flexibly almost anywhere in Perl code. The semantics of lexical variable and lexical pragmata cleanly defines that the lexical effect starts at the next statement after the declaration, and that it ends at exit from the block, without any of the surprises that we have seen in some other languages. The shadowing effect of lexical variables in inner scopes is easily understandable and consistent across all higher scopes, including the next englobing lexical scopes and the global module scope.

What a beautiful language design !

The next post will be about dynamic scoping through the local keyword - another, complementary way for temporarily changing the behaviour of the interpreter.

About the cover image

The picture is an excerpt from the initial movement of Verdi's Requiem, at a place where Verdi shadows several characteristics of the movement : for a short while, the orchestra stays still, leaving the choir a cappella, with a different speed, different tonality and different dynamics; then after this parenthesis, all parameters come back to their initial state, come prima as stated in the score.

Expressiveness of the Perl programming language

The collection of features in the Perl programming language is quite unique. Some of these features may seem surprising or even distasteful to people coming from other languages; yet their combination provides a fantastic toolbox for expressiveness in programming, where you can write not only effective, but also beautiful code.

True enough, expressivity can also be used for other purposes. Many years ago, a common entertainment game in the Perl community was the obfuscated Perl contest, where participants tried to exploit the most arcane parts of the language for producing illegible, yet working programs -- in the same spirit as today's brainrot videos! People were excited by the creativity potential of Perl. Other productions from that time also include Perl golf, Perl poetry and Perl haikus. But apart from fun, if your goals are conciseness, clarity of algorithms and long-term readability, you can use Perl too, with considerable benefits, because the code can be organized so that it closely reflects your thoughts. This is what I hope to show in this series.

Perl vs other languages : Compare facts, not opinions

Of course using the word "beautiful" in the title is opinionated ... but that is mainly to draw the reader's attention!

This series will try to focus on factual evidence, considering in turn various aspects of the Perl programming language and comparing those to similar or dissimilar constructs in other languages (mainly Python, Javascript and Java). This approach tries to stay away from the vast corpus of opinionated debates about Perl that can be found in many essays, posts or tweets. In most cases such discussions start with a general opinion and then use a few examples to illustrate the argument: under such conditions it is very unlikely that readers will be able to judge for themselves. A more profound comparison of languages requires time, requires details, and requires broad coverage; so here my intention is to go bottom-up, looking at many features of different granularity, so that you can decide for yourself if you find them indeed "beautiful".

What about Raku ?

Perl has a sister language now named Raku, formerly known as Perl6. Raku took a very long time to emerge, after a huge participative design effort that carefully discussed weaknesses of Perl and weighed the pros and cons of mechanisms to be incorporated into the new language. Since it was a fresh start, without backwards compatibility constraints, it was possible to freely choose what to keep and what to change.

The result is more than beautiful, it is awesome and brings Perl's spirit to another dimension ... but unfortunately Raku is still in a niche much smaller than Perl. Raku really deserves to get a larger audience, and has very qualified advocates for that; but that's not the purpose of the present series, which focuses on Perl5.

Technical notes about Perl fragments in the series

Upcoming posts will discuss various Perl features, in no particular order; some topics are small details, others are fundamental mechanisms.

Code fragments will be written in modern Perl, namely version 5.42.0, of course with pragmata strict, warnings and utf8 always activated (even if not repeated in every code fragment). Subroutine declarations will always use signatures. Object-oriented programming will mainly use core classes, or sometimes Moose when more sophisticated mechanisms are needed. Yes, this is Perl, there are several ways to do objects !

About the author

After 10 years in academia doing research in the field of theory of programming languages, I spent the rest of my career in public administration, using Perl for more than 25 years for a large number of tasks ranging from small, one-shot migration scripts to large enterprise applications or complex data analysis tools. The majority of these applications are still in use today, maintained by a team (contrary to the saying that "Perl code is write-only"), and so rich in features that their replacement by other technology will probably not happen for a while. I have also authored 36 modules published on the Comprehensive Perl Archive Network (CPAN).

My knowledge of Java and of Python is mostly theoretical, not consolidated by practical experience, so I apologize in advance if I write inaccurate statements about these languages -- please correct me if this is the case.

About the cover image

The picture shows the initial pages of Johann Sebastian Bach's motet Singet dem Herrn ein neues Lied. In the second half of the XVIIIth century, Bach's style went largely out of fashion because it was considered too complex, with an "excess of art that altered the beauty of the music". Common taste at that time preferred the galant style, with simpler melodies and reduced polyphony. Bach remained appreciated by a small circle of connoisseurs, though.

In 1789 when Mozart - already in his maturity - first heard the motet "Singet dem Herrn", he showed great enthusiasm, eagerly asking "what is this" and keen to study "something new at last from which he could learn". After this episode it took another 50 years before Bach's music started a slow revival, finally leading to his current recognition as one of the greatest composers of all times.

So Bach is here to remind us that notions of beauty and expressivity, and their relations with complexity or simplicity, may evolve over time!

Acknowledgements

Many thanks to Boyd Duffee, Matthew O. Persico and Marc Perry who took time to review this material before publication.

Cheers to Perl!

Weekly Challenge 359

Each week Mohammad S. Anwar sends out The Weekly Challenge, a chance for all of us to come up with solutions to two weekly tasks. My solutions are written in Python first, and then converted to Perl. It's a great way for us all to practice some coding.

Challenge, My solutions

Task 1: Digital Root

Task

You are given a positive integer, $int.

Write a function that calculates the additive persistence of a positive integer and also return the digital root.

• Digital root is the recursive sum of all digits in a number until a single digit is obtained.
• Additive persistence is the number of times you need to sum the digits to reach a single digit.

My solution

Starting this week, I'm going to turn off VS Code Copilot when completing the challenge. As AI takes over the task of code completion, it's still a good exercise to do things by hand so as I don't depend on it.

For this tasks, I create two variables. The first is called persistence and starts at 0. The second variable is called digital_root and starts with the supplied integer. This is called number in Python, as int is a reserved word.

I then have a loop that continues until digital_root is a single digit. Breaking down the line:

• str(digital_root) will convert the digital_root integer to a string
• d for d in will convert each character to a single digit
• map(int(... will convert the digit back to an integer
• sum(... will add the single digit

I also increment the persistence value by one. I end by returning the two values. The main function is responsible for displaying the text as per the examples.

def get_digital_root(number: int) -> tuple[int, int]:
    if number <= 0:
        raise ValueError("You must provide a positive integer")

    persistence = 0
    digital_root = number

    while len(str(digital_root)) > 1:
        digital_root = sum(map(int, (d for d in str(digital_root))))
        persistence += 1

    return persistence, digital_root

The Perl solution follows the same logic. Perl doesn't make a distinction between string and integers (yes, there are some exceptions to this rule), but does require the split function to break the digital_root value into individual digits.

use List::Util 'sum';

sub main ($int) {
    my $persistence  = 0;
    my $digital_root = $int;

    # Keep iterating until we have a single digital
    while ( length($digital_root) > 1 ) {
        $digital_root = sum( split( //, $digital_root ) );
        $persistence++;
    }

    say "Persistence  = $persistence";
    say "Digital Root = $digital_root";

Examples

$ ./ch-1.py 38
Persistence  = 2
Digital Root = 2

$ ./ch-1.py 7
Persistence  = 0
Digital Root = 7

$ ./ch-1.py 999
Persistence  = 2
Digital Root = 9

$ ./ch-1.py 1999999999
Persistence  = 3
Digital Root = 1

$ ./ch-1.py 101010
Persistence  = 1
Digital Root = 3

Task 2: String Reduction

Task

You are given a word containing only alphabets.

Write a function that repeatedly removes adjacent duplicate characters from a string until no adjacent duplicates remain and return the final word.

My solution

This is a duplicate (albeit slightly re-worded) of the first task in week 340. Therefore I copy and pasted that code, and renamed the function. See my original blog post on how this was solved.

Examples

$ ./ch-2.py aabbccdd
""

$ ./ch-2.py abccba
""

$ ./ch-2.py abcdef
"abcdef"

$ ./ch-2.py aabbaeaccdd
"aea"

$ ./ch-2.py mississippi
"m"