From 2015, but it came across my screen this morning. Seems like it is still relevant.

System setup for modern perl projects

Challenge 083

TASK #1 › Words Length

This is relatively straight forward task. I take the string and use the regex /^\s*\S+?\s+(.*)\s+\S+\s*$/ to strip out the first and last words (and any leading or trailing white space). I then strip out white-space from the string, and return the length.

Examples

» ./ch-1.pl "The Weekly Challenge"
6

» ./ch-1.pl "The purpose of our lives is to be happy"
23

TASK #2 › Flip Array

This is another task where I will be interested to see the other solutions Team PWC members offer up. Abigail made an excellent post last week explaining the number theory behind her work.

Firstly I've retrieve the values, and verified they are all positive numbers.

I then go through a loop from 0 to 2ⁿ-2 (2ⁿ-1 is all negative, and thus won't provide a non-negative sum). If the binary bit is true, then I flip the number. For example if the numbers are 2 4 6 8 and the loop count is 5 (binary 101), then the combination I am using is 2 -4 6 -8.

Once I have the combination in the loop, I sum the numbers. If the sum is less than 0, I proceed to the next iteration. If the solution is better than any previous numbers (either a lower sum, or the same sum with a lower number of flipped values), then I record it.

Once the loop is complete, I then present the result.

This solution requires 2ⁿ-1 calculations. So even for 10 numbers, there is only 1023 iterations through the main loop. There may be more efficient ways to solve this challenge. I guess I'll find out during this week.

Examples

» ./ch-2.pl 3 10 8
RESULT IS: 1 (-10 3 8)

» ./ch-2.pl 12 10 2
RESULT IS: 1 (-12 2 10)

Hi there

Are you active on Facebook? If yes then you should join the weekly "Perl Town Hall" host by Will Braswell. It is weekly live video session every Friday. I do join the show whenever I get time. Because of time difference, it usually comes live midnight Friday. Credit goes to Will for putting up such a nice show mix of Perl news. I just hope if it is available for wider audience outside of Facebook. Feel free to join the Facebook group "Perl Programmers", so that you don't miss the fun every week. The best part is you can interact with him live during the show.

While talking about social platform, how can I miss "Perl Maven", page on LinkedIn. It is run and managed by the Gabor Szabo. Please do join and follow the page for more Perl related posts and videos.

If you follow The Weekly Challenge then why not check out the weekly review of Perl and Raku by Colin Crain and Andrew Shitov respectively.

I would like to take this opportunity to thank you for the support and encouraging emails. It means a world to me. This is my 63rd issue of newsletter. Thank you, Gabor, for guiding me on this journey.

Last but not the least, you take care of yourself and your loved ones. Enjoy rest of the newsletter.

These are some answers to the Week 82 of the Perl Weekly Challenge organized by Mohammad S. Anwar

On Friday, Oct. 16, 2020 around 5:00 p.m., an awful terrorist attack was perpetrated in my home-town of Conflans Sainte-Honorine in France (35,000 inhabitants), the city where I live and of which I am a city councilor. A secondary school teacher was beheaded by a crazy religious extremist who coudn’t accept that teacher’s defense of the freedom of speech. This is a terrible shock to all my fellow citizens, to the teachers and pupils of that school, and to my fellow members of the city council. Because of that, I did not have time to complete the second task of this challenge (although it was almost complete) and my blog post on the first task will be shorter than what I wanted to make. I actually considered not publishing a blog post this week, but I have written at least one blog post for every single Perl Weekly Challenge since the very beginning, I certainly do not want a madman to prevent me from continuing this uninterrupted series of blogs on PWC. And I also don’t want to leave my friend Mohammad S. Anwar alone in the cold.

Common Factors

You are given 2 positive numbers $M and $N.

Write a script to list all common factors of the given numbers.

Example 1:

Input:
    $M = 12
    $N = 18

Output:
    (1, 2, 3, 6)

Explanation:
    Factors of 12: 1, 2, 3, 4, 6
    Factors of 18: 1, 2, 3, 6, 9

Example 2:

Input:
    $M = 18
    $N = 23

Output:
    (1)

Explanation:
    Factors of 18: 1, 2, 3, 6, 9
    Factors of 23: 1

I wanted to give some more explanations, but the short version is that the common factors of two numbers are the factors of the greatest common divisor (GCD) of the two input numbers. So we will compute the GCD of the two numbers and find its divisors.

Common Factors in Raku

Raku has a built-in operator, gcd, to compute the GCD of two numbers.

The code is thus quite simple:

sub common_factors (Int $a, Int $b) {
    my $gcd = $a gcd $b;
    return (1,) if $gcd == 1;
    return ($gcd,) if $gcd.is-prime;
    return (1..$gcd).grep($gcd %% *).unique;
}
my @result = common_factors 12, 18;
say @result;

Output:

raku common_factors.raku
[1 2 3 6]

Common Factors in Perl

Perl doesn’t have a built-in GCD routine. So we will write one using Euclid’s algorithm, or, rather, a modernized version of his algorithm using the % modulo operator, rather than subtraction in the original version of Euclid’s algorithm.

use strict;
use warnings;
use feature "say";

sub gcd {
    my ($i, $j) = sort { $a <=> $b } @_;
    while ($j) {
        ($i, $j) = ($j, $i % $j);
    }
    return $i;
}

sub common_factors {
    my ($i, $j) = @_;
    my $gcd = gcd ($i, $j);
    return (1) if $gcd == 1;
    my @factors = grep { $gcd % $_ == 0 } 1..$gcd; 
    my %unique = map {$_ => 1} @factors;
    return sort { $a <=> $b } keys %unique;
}
say join " ", common_factors @ARGV;

Output:

$ perl common_factors.pl 12 18
1 2 3 6

Wrapping up

The next week Perl Weekly Challenge will start soon. If you want to participate in this challenge, please check https://perlweeklychallenge.org/ and make sure you answer the challenge before 23:59 BST (British summer time) on Sunday, October 25, 2020. And, please, also spread the word about the Perl Weekly Challenge if you can.

The Perl Foundation Standards of Conduct sets Standards of Conduct for how our community interacts with others during the conference and in other TPF spaces. The team that enforces these standards is called the Community Affairs Team (CAT). That team was available to respond to incident reports during the Conference in Cloud in June, 2020 (aka The Perl and Raku Conference).

In order to be open and transparent about this process, we have prepared the following report of incidents during the conference.

There was one incident handled by the team for this conference. Slides for a talk were reported as containing inappropriate material. The CAT reviewed the recording of the talk and agreed that this was the case. The CAT talked with the speaker, explaining the issue and the consequences. The speaker was given a final warning: if there is another incident by the speaker, they will be prohibited from speaking at TPF events for at least one year.

The Grants Committee has concluded voting on the September 2020 round. Two grant requests were submitted:

General Perl OpenAPI Validator / Interpreter (USD 8,370)

VOTING RESULTS: Not Approved. 1 YES vote (2 points), 5 NO votes, 3 abstentions

Some comments and concerns brought up by the Grants Committee members include: * Would like to see more tangible deadlines * A better analysis of why this is needed * Timeframe seems overly optimistic

Ć-to-Raku translator (USD 8,192)

VOTING RESULTS: Not Approved. 0 YES votes, 5 NO votes, 4 abstentions

While this seems like an interesting project, is wasn't exactly clear what needs this is filling for the Raku community.

We accept proposals throughout the year; our next round of review will begin in early November. You can submit proposals at any time.

If you want to help with funding and increase our budget, please visit our donations page. We sincerely appreciate all the donors which make the grant program possible. If you donate, please take advantage of your employers' matching donation program.

As always, thanks to our donors, both large and small, who support this program to give back to the community.

Part 1

You are given 2 positive numbers $M and $N. Write a script to list all common factors of the given numbers.

Solution

use strict;
use warnings;
##
# You are given 2 positive numbers $M and $N.
# Write a script to list all common factors of the given numbers.
##
sub factor{
    my($n) = @_;
    my @factors = (1);
    foreach my $j (2..sqrt($n)){
        push @factors, $j if $n % $j == 0;
        push @factors, ($n / $j) if $n % $j == 0 && $j ** 2 != $n;
    }    
    return @factors;  
}

sub common_factors{
    my($m, $n) = @_;
    my @common_factors = grep { my $f = $_; grep { $f == $_ } @{$n}} @{$m};
    return @common_factors;
}


MAIN:{
    my $M = 12;
    my $N = 18;
    my @m_factors = factor($M);
    my @n_factors = factor($N);
    print "(" . join(",", common_factors(\@m_factors, \@n_factors)) . ")\n";
}

Sample Run

$ perl perl/ch-1.pl
(1,2,6,3)

Notes

I have used sub factor previously, back in Challenge 008. The most interesting thing in this solution is probably the nested grep’s. In order to nest them properly you need to create a local variable to hold the element being examined in the outer grep block. Here I use $f. Although we need just two grep’s here this trick can be used to nest them more deeply.

Part 2

You are given 3 strings; $A, $B and $C. Write a script to check if $C is created by an interleaving of $A and $B. Print 1 if check is success otherwise 0.

use strict;
use warnings;
##
# You are given 3 strings; $A, $B and $C.
# Write a script to check if $C is created by interleave $A and $B.
# Print 1 if check is success otherwise 0.
##
sub find_remove{
    my($s, $x) = @_;
    my $i = index($s, $x);
    if($i != -1){
        substr $s, $i, length($x), "";
        return $s;
    }
    return undef;
}
MAIN:{
    my $A = "XY";
    my $B = "X";
    my $C = "XXY";
    my $s = find_remove($C, $A);
    if($s && $s eq $B){
        print "1\n";
        exit;
    }
    else{
        $s = find_remove($C, $B);
        if($s && $s eq $A){
            print "1\n";
            exit;
        }
    }
    print "0\n";
}

Sample Run

$ perl perl/ch-2.pl
1

Notes

I believe this is the most straightforward way of tackling this problem. By checking for both $A and $B as substrings and removing them if found we can determine if there was an interleaving by checking to see if the other remains.

I am writing this blog post to hand over all my programming books, (mostly perl) to any person that would like to use them.

This includes about a dozen O'rielly perl books (including the camel book) as well as other classics like:

The C programing language by K+R
TCP/IP by Comer
Design Patterns by GHJV

all free, but must take the whole box.

Contact me at freddo411 (at) gmail.

I am located in the Seattle area.

This blog post describes a common task my colleagues ask often about repeating a dynamic string in a defined token and adding some or, and, = in between, plus finishing smartly.

I like to use the Perl's __DATA__ token at the end of my scripts for this. The __DATA__ token strength is to make possible to « "embed” a file inside a Perl program then read it from the DATA filehandle ». It saves you the creation and opening of a real file and is very handy for quick prototypes and tests.

#!/usr/bin/env perl
use strict;
use warnings;

# Your script here

# Everything under is considered as
# the end of the code
__DATA__
a
lot
lot
of
stuff
here
...

A common practice is to load those data to an array by treating them as a file handle:

my @lines = <DATA>;

But the values would include carriage returns, what you obviously don't want. I used two solutions for this:

my @lines;
push @lines,
  split while <DATA>;

This is quite readable and self-explanatory (remember Perl a natural language, it was created by a linguist). Feel free to comment if something is unclear so I could improve the post.

Ok, I have to admit a little secret:

push my @lines,
  uniq split while <DATA>;

... without the pre-declaration of @lines does the same. I had to counter check it worked, but as often with Perl, when you spontaneously think of something silly, it actually works naturally (I have to admit it sometimes looks like a miracle).

If you want uniq values (you surely do), one way is to use the core module List ::Util:

use List::Util qw(uniq);

push my @lines,
  uniq split while <DATA>;

Another way to do it is always possible:

chomp( my @lines = uniq <DATA> );

I actually prefer this list context solution, for it's shortness, dunno which one is the more readable, and it is good to choose the readable way.

Let's say you want to generate a series of or for your colleagues or customers. We are actually doing a super advanced language generation thing here:

#!/usr/bin/env perl
use strict;
use warnings;
use List::Util qw(uniq);

chomp( my @lines = uniq <DATA> );

for ( @lines ) {
  # $_ is the current loop element
  print generate_string( $_ );
  # $lines[-1] is the last array element
  if ( not $_ eq $lines[-1] ) {
    print ' or ';
  } else {
    print "\n";
  }
}

sub generate_string {
  return 'line == "' . shift . '"';
}

__DATA__
a
lot
lot
of
stuff
here
...


$ perl lines.pl
line == "a' or line == "lot' or line == "of' or line == "stuff' or line == "here' or line == "...'

Lots of other solutions exist, check the Perl one-liners thing that allow to learn a lot more about those kind of practices.

The quantities of cools things you can do inside this loop is infinite, from log parsing to generating code or data munging, thanks to the kindness of Perl.

References

• Read a file into an array using Perl
• Stupid DATA Tricks
• This is actually a crosspost of the same article on dev.to

Note

I wrote this because my memory is awful and I was tired of always searching for the exact syntax of the __data__ token to array process. Hope it will help all kinds of people including me when I type it in a search engine.

During the work for this grant, the author (Ben Davies) stumbled upon some issues (best described in #111 Problem Solving) that block further progress on the existing grant as it is currently written. At Ben’s request, the grant as it exists now is being cancelled, to be replaced with a new proposal to address the issues he encountered, as well as revised proposals to cover the remaining ideas from the original grant. The new grants will follow the normal process for funding.

Thanks for your hard work, Ben - looking forward to your future proposals!

If you're anything like me you're a lazy bugger and prefer to write as little code as possible, instead relying on libraries that other people have written. This has several advantages: it's less work for you, the code has already been tested (you hope!) and debugged, someone else maintains it for you, and so on. The disadvantage is that those other people might change things, either accidentally or on purpose, and break your code. So if you're going to use other peoples' libraries it's a good idea to know what libraries you're actually using.

Easy to figure out, right? Just remember what you're using!

Oh if only it were that simple. You see, all those other people whose code you're using are just as lazy as you are, and they're all using third parties' libraries, and in turn those are using yet more libraries, and so on. As a learnèd sage once wrote:

Bigger fleas have littler fleas upon their backs to bite 'em,
And little fleas have lesser fleas, and so on, ad infinitum.

Many years ago I wrote a script ¹ which shows the entire dependency tree for a module on the CPAN. Those trees grow surprisingly big surprisingly fast. For one of mine, it has 1 direct dependency, 8 second level dependencies, another 13 at the third level, and so it continues for a grand total of 41 ² dependencies.

Params::Validate::Dependencies::all_or_none_of (dist: D/DC/DCANTRELL/Params-Validate-Dependencies-all_or_none_of-1.01.tar.gz)
  Params::Validate::Dependencies (dist: D/DC/DCANTRELL/Params-Validate-Dependencies-1.40.tar.gz)
    Clone (dist: A/AT/ATOOMIC/Clone-0.45.tar.gz)
      B::COW (dist: A/AT/ATOOMIC/B-COW-0.004.tar.gz)
    Data::Domain (dist: D/DA/DAMI/Data-Domain-1.07.tar.gz)
      Date::Calc (dist: S/ST/STBEY/Date-Calc-6.4.tar.gz)
        Bit::Vector (dist: S/ST/STBEY/Bit-Vector-7.4.tar.gz)
          Carp::Clan (dist: E/ET/ETHER/Carp-Clan-6.08.tar.gz)
      List::MoreUtils (dist: R/RE/REHSACK/List-MoreUtils-0.428.tar.gz)
        Exporter::Tiny (dist: T/TO/TOBYINK/Exporter-Tiny-1.002002.tar.gz)
        List::MoreUtils::XS (dist: R/RE/REHSACK/List-MoreUtils-XS-0.428.tar.gz)
        Test::LeakTrace (dist: L/LE/LEEJO/Test-LeakTrace-0.16.tar.gz)
      Scalar::Does (dist: T/TO/TOBYINK/Scalar-Does-0.203.tar.gz)
        Test::NoWarnings (dist: A/AD/ADAMK/Test-NoWarnings-1.04.tar.gz)
        Test::Requires (dist: T/TO/TOKUHIROM/Test-Requires-0.11.tar.gz)
        Type::Tiny (dist: T/TO/TOBYINK/Type-Tiny-1.010006.tar.gz)
        URI (dist: O/OA/OALDERS/URI-1.76.tar.gz)
          Test::Needs (dist: H/HA/HAARG/Test-Needs-0.002006.tar.gz)
... and so on

But even more important than knowing what's in your dependency tree is knowing when that tree changes, so that you know when you need to re-test your code to make sure no-one else broke it. Well, I've finally gone and created a tool for that, after having it on my to-do list for over a decade (did I mention above that I'm very lazy?).

It maintains a little database in ~/.cpandeps-diff of what a module's dependencies are, and every time you run it without any arguments it compares the dependencies it already knows about with what's on the CPAN, spitting out a side-by-side diff. Invoke it daily from your crontab and you'll get a daily email with any changes.

Start by telling it what modules you want to get reports about:

$ cpandeps-diff add CPAN::FindDependencies
$ cpandeps-diff add Sub::WrapPackages
$ cpandeps-diff add Number::Phone

Each time it will go away to the CPAN, find what all the dependencies are, and store them in a file in that directory. You will also notice a cache directory, which is used to minimise traffic to the CPAN mirrors.

You can list which modules it knows about, and delete any that you are no longer interested in:

$ cpandeps-diff list
CPAN::FindDependencies
Number::Phone
Sub::WrapPackages

$ cpandeps-diff rm Sub::WrapPackages

And you can get a report on an individual module:

$ cpandeps-diff report Number::Phone
Differences found in dependencies for Number::Phone:
+---+-----------------------------------------------+-----------------------------------------------+
|  3|E/ET/ETHER/Class-Method-Modifiers-2.13.tar.gz  |E/ET/ETHER/Class-Method-Modifiers-2.13.tar.gz  |
|  4|E/ET/ETHER/File-ShareDir-Install-0.13.tar.gz   |E/ET/ETHER/File-ShareDir-Install-0.13.tar.gz   |
|  5|E/ET/ETHER/Try-Tiny-0.30.tar.gz                |E/ET/ETHER/Try-Tiny-0.30.tar.gz                |
*  6|E/EX/EXODIST/Test-Exception-0.42.tar.gz        |E/EX/EXODIST/Test-Exception-0.43.tar.gz        *
|  7|H/HA/HAARG/Moo-2.004000.tar.gz                 |H/HA/HAARG/Moo-2.004000.tar.gz                 |
|  8|H/HA/HAARG/Role-Tiny-2.001004.tar.gz           |H/HA/HAARG/Role-Tiny-2.001004.tar.gz           |
...

You can see in that report that a new version of Test-Exception has been released, and so I should make sure that the code which depends on it still works. If it turns out that that broke my code, that's annoying, but at least I found out about the problem before users started reporting bugs.

1. I also created a website, originally based on largely the same code, which also shows test results for all dependencies and which has bit-rotted and become overloaded to the extent that it is no longer usable. Sorry about that. ↩

2. or if you're using a really old version of perl, and so need upgraded versions of libraries distributed with the interpreter, you can more than double that. ↩

Before leaving on his much deserved break, Jonathan was able to carry forward the performance work ahead in September. (Aside from his work on the RakuAST grant).

`` 01:20 Better specialize and JIT boolifcation of Num, so we produce rather better machine code for it 11:35 Continue work on the new dispatch mechanism, with a focus on dispatch resumption. Implemented a means to indicate what state is needed to resume a dispatch (for example, withcallsamein a method, this is the original dispatch arguments along with the method name). Multiple dispatchers within a single dispatch can decide what they need to retain (for example, if a method dispatch does a multi dispatch to a wrapped method, all 3 need state for handlingcallsame`). The mechanism is designed to minimize costs in the common case that this information is ultimately not needed. Also some initial wiring up of resume dispatch callbacks, where the resumption logic will take place.

Total time worked: 12 hours 55 minutes ```

GraphViz2::Parse::ISA can visualize N Perl class hierarchies as a graph.

I used the code from the synopsys on the files that come with the distribution:

This blog post describes a common task my colleagues ask often about repeating a dynamic string in a defined token and adding some or, and, = in between, plus finishing smartly.

I like to use the Perl's __DATA__ token at the end of my scripts for this. The __DATA__ token strength is to make possible to « "embed” a file inside a Perl program then read it from the DATA filehandle ». It saves you the creation and opening of a real file and is very handy for quick prototypes and tests.

#!/usr/bin/env perl
use strict;
use warnings;

# Your script here

# Everything under is considered as 
# the end of the code
__DATA__
a
lot
lot
of
stuff
here
...

A common practice is to load those data to an array by treating them as a file handle:

my @lines = <DATA>;

But the values would include carriage returns, what you obviously don't want. I used two solutions for this:

my @lines;
push @lines, 
 split while <DATA>;

This is quite readable and self-explanatory (remember Perl a natural language, it was created by a linguist). Feel free to comment if something is unclear so I could improve the post.

Ok, I have to admit a little secret:

push my @lines, 
  uniq split while <DATA>;

... without the pre-declaration of @lines does the same. I had to counter check it worked, but as often with Perl, when you spontaneously think of something silly, it actually works naturally (I have to admit it sometimes looks like a miracle).

If you want uniq values (you surely do), one way is to use the core module List ::Util:

use List::Util qw(uniq);

push my @lines, 
  uniq split while <DATA>;

Another way to do it is always possible:

chomp( my @lines = uniq <DATA> );

I actually prefer this list context solution, for it's shortness, dunno which one is the more readable, and it is good to choose the readable way.

Let's say you want to generate a series of or for your colleagues or customers. We are actually doing a super advanced language generation thing here:

#!/usr/bin/env perl
use strict;
use warnings;
use List::Util qw(uniq);

chomp( my @lines = uniq <DATA> );

for ( @lines ) {
  # $_ is the current loop element
  print generate_string( $_ );
  # $lines[-1] is the last array element
  if ( not $_ eq $lines[-1] ) {
    print ' or ';
  } else {
    print "\n";
  }
}    

sub generate_string {
  return 'line == "' . shift . '"';
}

__DATA__
a
lot
lot
of
stuff
here
...

$ perl lines.pl
line == "a' or line == "lot' or line == "of' or line == "stuff' or line == "here' or line == "...'  

Lots of other solutions exist, check the Perl one-liners thing that allow to learn a lot more about those kind of practices.

The quantities of cools things you can do inside this loop is infinite, from log parsing to generating code or data munging, thanks to the kindness of Perl.

References

• Read a file into an array using Perl
• Stupid DATA Tricks

Note

I wrote this because my memory is awful and I was tired of always searching for the exact syntax of the __data__ token to array process. Hope it will help all kinds of people including me when I type it in a search engine.

This is a quick reminder that the quarterly San Diego Perl Mongers meeting will be occurring on Tuesday evening, starting at our normal time of 7 PM PDT.  As has been for the last several meetings, we’re going to meet online, as in-person meetings are discouraged, and online meetings seem to be a bit more popular.

Here’s the Meeting ID: 851 5455 5818

Here’s the Zoom link: https://us02web.zoom.us/j/85154555818

The passcode can be acquired by running the short Perl program:

sub sum {
    $a = shift();
    $a += shift() while @_;
    return $a;
}
@n = ( 3 ); END{ print @n, "\n"; }
push @n, sum( $n[0], 0+@n );
push @n, sum( $n[0], $n[-1] ) for @n[0,1];
push @n, sum( @n ) / $n[0];

We’ll start the conversation by checking in with everyone, and seeing if there are any questions. Following that, I’m trying to work up a short presentation. Otherwise, it looks to be a mostly social meeting.

We’re looking forward to an enjoyable evening with you all!

One of the most common tasks in programming is receiving arguments within a function. In Perl, historically, this task has taken on a number of forms, and it has usually looked like this:

sub foo {
  my $arg1 = shift;
  my $arg2 = shift;

  return $arg1 + $arg2;
}

As many Perlers already know, the shift statement is just short-hand for shift @_, which takes the first item off of the special @_ list.

There have been other variations on this theme:

# Named arguments
sub foo {
  my %args = @_;
  my $arg1 = delete $args{arg1};
}

# Named arguments with a hashref
sub foo {
  my ($args) = @_;
  my $arg1 = delete $args->{arg1};
}

# Positional arguments in a single assignment
sub foo {
  my ($arg1, $arg2) = @_;
}

This is a marvelous amount of flexibility, and many folks have found incredible ways of making this work to their advantage. It is a very different way of doing function arguments, though, and if you've been doing Perl long enough you've already spotted that all of the samples above will have that most beloved of warnings, Use of uninitialized value.

Of course, you could use a default:

sub foo {
  my $foo = shift || 1;
}

but what if you want your caller to get an error if they fail to give you enough arguments? Well, you could use function prototypes, like this:

sub foo($) {
  my $foo = shift;
}

If you did it this way, code calling this function as foo() would get the Not enough arguments for main::foo error. You then would still be able to handle arguments the same way as you normally would after using the prototype, though.

However, the Perl community has long discouraged the use of prototypes, as the syntax is a little hairy and it's not always clear what the developer was trying to do when they started using them.

Fast-forward to Perl 5.20 and now you have signatures!

Quick note: Brian D Foy did a much better write-up here than I'm going to. I'm only going to tell you why I love it.

With function signatures, I can remove boilerplate (many will know that this is one of the time-honored traditions of Perl programming, dispensing with boilerplate) and instead I can focus on the intention of my code.

sub some_function($a, $b) {
  return $a + $b;
}

I can even do this with closures:

my $some_closure = sub ($a, $b) {
  return $a + $b;
}

Right; I have a pretty syntax that I like, whoop-dee-doo for me, right? Well, here's why you like it.

Remember that Mojolicious thing that I've ranted about before? That you should be loving right now? Or any of those event-based modules that you've been loving? This is where the rubber and road fall in love and get a puppy together.

# This is pseudo-code that might run!
use Mojo::Promise;

sub long_running_operation() {
  state $cached = [];

  return (scalar @{$cached} > 0)
    ? Mojo::Promise->resolve( shift @{$cached} )
    : Mojo::Promise->new( sub ($resolve, $reject) {
      my $retval = eval {
        perform_slow_lookup();
      }; if ($@ or not $retval) {
        return $reject->($@);
      }

      return $resolve->($retval);
    })->then( sub ($results = []) {
      $cached = { map { $_->{name} => $_ } @{$results} };
      return Mojo::Promise->resolve( shift @{$cached}; );
    });
}

Oh, did I say I was going to make you fall in love with Mojo::Promise, too? My bad, I should have warned you.

So now, with function signatures you have eliminated a remarkable amount of boilerplate here (about four lines in just this snippet), which reduces your surface area for bugs while also making your code smaller and more expressive.

Without signatures, this code would look something like this:

# This is pseudo-code that might run!
use Mojo::Promise;

sub long_running_operation() {
  state $cached = [];

  return (scalar @{$cached} > 0)
    ? Mojo::Promise->resolve( shift @{$cached} )
    : Mojo::Promise->new( sub {
      my ($resolve, $reject) = @_;
      my $retval = eval {
        perform_slow_lookup();
      }; if ($@ or not $retval) {
        return $reject->($@);
      }

      return $resolve->($retval);
    })->then( sub {
      my $results = shift || [];
      $cached = { map { $_->{name} => $_ } @{$results} };
      return Mojo::Promise->resolve( shift @{$cached}; );
    });
}

See? It's not the end of the world, but it does take away some of the clarity in the code by adding boilerplate lines that you have to visually parse through in order to see what you're trying to do.

Anyway, that's just what crossed my mind while I'm writing more things to trawl GitHub.

Have a great day!

Challenge 082

For both of these challenges, even though the requirement is matching two values, my solutions will work with two or more values.

TASK #1 › Common Factors

While there is probably a Math:: function that generates the factors of a given number, I'm of the belief that it defeats the point of the weekly challenge. I tend to only use List::Util functions, which is part of Perl core.

This is a relatively straight forward task. I create a loop from 1 to the minimum of the supplied values (any value greater than this won't be a factor of the minimum number). I then add that number to the result if the division of all inputs by the number results in no remainder.

Examples

» ./ch-1.pl 12 18
1, 2, 3, 6

» ./ch-1.pl 18 23
1

TASK #2 › Interleave String

I'm really looking forward to seeing the solutions to this task by other Team PWC members. I'm not sure my solution is the best one, but it works. Let me also sing the praises of the map function. The best thing since sliced bread.

I start by short circuiting the code to return 0 if the length of the target is not the length of the other strings. No solution would be possible in this scenario.

The approach that I took was having any array of arrays called @remaining. Each row represents the remaining letters available from each string. It starts with one row representing all the strings.

I then work through each letter of the target string. Using the map function, I take the first letter off any string that starts with that letter.

For example, in the first example ('XY X XXY'), @remaining starts with (('XY', 'X')). After the first run, @remaining becomes (('Y', 'X'), ('XY', '')) as both strings start with an 'X'. The second iteration would result in (('Y', ''), ('Y', '')) removing the second 'X'. The final run would result with (('', ''), ('', '')) removing the Y in the last character.

If at any point remaining has no values, we return 0. Once we have exhausted all letters, we know their is a solution, so return 1.

Examples

» ./ch-2.pl XY X XXY
1

» ./ch-2.pl XXY XXZ XXXXZY
1

» ./ch-2.pl YX X XXY
0

Part 1

You are given 2 strings, $A and $B. Write a script to find out common base strings in $A and $B.

Solution

use strict;
use warnings;
##
# You are given 2 strings, $A and $B.
# Write a script to find out common base strings in $A and $B.
## 
use boolean;

sub contains{
    my($s0) = @_;
    return sub{
        my($s) = @_;
        return [true, $s0] if($s =~ m/^($s0)+$/g);
        return [false, $s0];
    }
}

sub make_checks{
    my($s) = @_;
    my @letters = split(//, $s);
    my @checks;
    for my $i (0 .. (int(@letters/2 - 1))){
        push @checks, contains(join("", @letters[0 .. $i]));
    }
    return @checks;
}

MAIN:{
    my($A, $B);
    #$A = "aaaaaaa";
    #$B = "aaaaaaaaaaaaaaaaaa";
    $A = "abcdabcd";
    $B = "abcdabcdabcdabcd";
    my @checks = make_checks($A);
    for my $check (@checks){
        if($check->($A)->[0] && $check->($B)->[0]){
            print $check->($A)->[1] . "\n";
            exit;
        }
    }
}

Sample Run

$ perl perl/ch-1.pl
abcdabcd, abcdabcdabcdabcd --> (abcd abcdabcd)
aaa, aa --> (a)

Notes

I used a similar technique to what I used in Challenge 003 and Challenge 004 where I create an array of anonymous functions which each check different substrings. The functions are created using what is called currying whereby we pass in a parameter (or multiple parameters if we needed to!) to a function which creates a closure around those parameters and returns a new function. This is, I admit, not all that necessary! I repeat the method though out of a sense of nostalgia. We are now on Challenge 081!

Part 2

You are given file named input. Write a script to find the frequency of all the words. It should print the result as first column of each line should be the frequency of the word followed by all the words of that frequency arranged in lexicographical order. Also sort the words in the ascending order of frequency.

use strict;
use warnings;
##
# You are given file named input.
# Write a script to find the frequency of all the words.
# It should print the result as first column of each line should be the frequency of the 
# word followed by all the words of that frequency arranged in lexicographical order. Also 
# sort the words in the ascending order of frequency.
##
MAIN:{
    my %counts;
    my %count_words;
    my $s;
    {    local $/;
         $s = ;
    }
    $s =~ s/'s//g;
    $s =~ tr/."(),//d;
    $s =~ tr/-/ /;
    my @words = split(/\s+/, $s);
    for my $word (@words){
        $counts{$word}++;
    }
    for my $k (keys %counts){
        my $count = $counts{$k};
        push @{$count_words{$count}}, $k;
    }
    for my $k (sort keys %count_words){
        print $k . "\t" . join(" ",  sort {$a cmp $b} @{$count_words{$k}}) . "\n";
    }
}


__DATA__ 
West Side Story

The award-winning adaptation of the classic romantic tragedy "Romeo and
Juliet". The feuding families become two warring New York City gangs,
the white Jets led by Riff and the Latino Sharks, led by Bernardo. Their
hatred escalates to a point where neither can coexist with any form of
understanding. But when Riff's best friend (and former Jet) Tony and
Bernardo's younger sister Maria meet at a dance, no one can do anything
to stop their love. Maria and Tony begin meeting in secret, planning to
run away. Then the Sharks and Jets plan a rumble under the
highway--whoever wins gains control of the streets. Maria sends Tony to
stop it, hoping it can end the violence. It goes terribly wrong, and
before the lovers know what's happened, tragedy strikes and doesn't stop
until the climactic and heartbreaking ending.

Sample Run

$ perl perl/ch-2.pl
1   But City It Jet Juliet Latino New Romeo Side Story Their Then West York adaptation any anything at award away become before begin best classic climactic coexist control dance do doesn't end ending escalates families feuding form former friend gains gangs goes happened hatred heartbreaking highway hoping in know love lovers meet meeting neither no one plan planning point romantic rumble run secret sends sister streets strikes terribly their two under understanding until violence warring what when where white whoever winning wins with wrong younger
2   Bernardo Jets Riff Sharks The by it led tragedy
3   Maria Tony a can of stop
4   to
9   and the

Notes

I have to admit that sometimes, even after many years of using Perl, that if I don’t use a certain feature often enough that I end up getting a little surprised. The surprise here is that Perl is clever enough to know that if I am trying a push onto a hash value which is undef, such as when I first do a push @{$count_words{$count}}, $k; for a new value of $k, that a new array is created. No need to check for undef and create a new one manually. This is called autovivification and while a very common thing in Perl for whatever reason it managed to catch me a little by surprise this time around. Probably due to my working a lot in other languages recently that don;t have this feature! Gabor has a nice writeup on autovivification for anyone interested in reading more.

Nicolas R. has shared Jason McIntosh's updates for September for the Establishing Documentation Standards for the Perl 7 Era grant:

Major steps towards my goal accomplished in September:

• Announced my starting this project on #p5p on Freenode, and collecting some basic information there

• Gathering Perl’s documentation style & policy information (of which there ain’t much, of course, hence the present project)

• Touring other FOSS languages and technologies with their own documentation sub-projects, and comparing them to what Perl has. Projects studied so far include Python, Rust, Raku, and Linux's "man-pages”.

• Have started to introduce myself to some of the above projects’ online communities, looking to make some contacts in their respective docs teams

Regarding the governance conversation started last month:

I’ve skimmed the ongoing discussion and action there, and see that the floor is shifting at this very moment regarding the way that Perl gets developed and maintained in the future. By the same token, I’m glad to see the possessors of the multiple visions for Perl’s future showing up to do the hard work of discussing and compromising and forging new, lasting agreements towards a set of common goals.

I intend to continue paying attention to this work, as it seems likely that I’ll want to work this nascent governing body into the delivery phase of my own project.

Tiny experiment to show "Hello World" with Mojolicious in Docker.

Image credit: Rob Nguyen on Flickr

I’ve previously written about Stupid Open Tricks, so know it’s time for some stupid DATA tricks. You probably already know that you can “embed” a file inside a Perl program then read it from the DATA filehandle. David Farrell wrote about this in Perl tokens you should know and he’s the one who reminded me about the curiousity that I’ll demonstrate here.

Anything after the __DATA__ line is not part of the program but is available to the program through the special DATA filehandle:

#!/usr/bin/perl
print "---Outputting DATA\n", <DATA>, "---Done\n";
__DATA__
Dog
Cat
Bird

The output shows each line after __DATA__:

---Outputting DATA
Dog
Cat
Bird
---Done

I typically go the other way by starting with a data file and adding a program to the top of it:

#!/usr/bin/perl
use v5.26;
use Text::CSV_XS;

my $csv = Text::CSV_XS->new;
while( my $row = $csv->getline(*DATA) ) {
	say join ':', $row->@[3,7];
    }
close $fh;

__DATA__
...many CSV lines...

This is the end, my friend, the END

You probably also know that you can use __END__ instead. I’m used to using that because it’s a holdover from Perl 4 and that’s where I first learned this:

#!/usr/bin/perl
print "---Outputting DATA\n", <DATA>, "---Done\n";
__END__
Dog
Cat
Bird

You get the same output:

---Outputting DATA
Dog
Cat
Bird
---Done

But now let’s get a little tricky. Define a package at the end of the program. This still uses __END__:

#!/usr/bin/perl
print "---Outputting DATA\n", <DATA>, "---Done\n";
package not::main;
__END__
Dog
Cat
Bird

Again, this outputs the same thing as before. Nothing surprising here, but the suspense must be building:

---Outputting DATA
Dog
Cat
Bird
---Done

Change that __END__ to __DATA__ and try again:

#!/usr/bin/perl
print "---Outputting DATA\n", <DATA>, "---Done\n";
package not::main;
__DATA__
Dog
Cat
Bird

Now you don’t see those lines:

---Outputting DATA
---Done

If you’ve read the documentation and cared about this sort of thing (or like me, forgotten it), you may have noticed that the DATA handle lives in the package that’s in scope at the end of the program:

Text after DATA may be read via the filehandle “PACKNAME::DATA”, where “PACKNAME” is the package that was current when the DATA token was encountered.

I can use the package specification to get the lines back:

#!/usr/bin/perl
print "---Outputting DATA\n", <not::main::DATA>, "---Done\n";
package not::main;
__DATA__
Dog
Cat
Bird

Now those lines are back:

---Outputting DATA
Dog
Cat
Bird
---Done

But what about the __END__? Well, that was a Perl 4 thing, before there were packages. Perl 5 added packages, then Perl 5.6 added the __DATA__ token. The __END__ kept doing what it was doing in the way it was doing it (package-less), and __DATA__ did something related by new:

For compatibility with older scripts written before DATA was introduced, END behaves like DATA in the top level script (but not in files loaded with “require” or “do”) and leaves the remaining contents of the file accessible via “main::DATA”.

Some other DATA tricks

There’s a few other interesting things you can do.

Program size

You can get the entire file size with the -s file test operator. The __DATA__ (or __END__) has to be there, but you don’t need any data after those tokens.

use v5.10;
my $size = -s DATA;
say "File size is $size";
__DATA__

The file size this reports includes everything in the file, not just the part before the end of processing.

use v5.10;
my $size = -s DATA;
say "File size is $size";

my $data = tell DATA;
say "Data starts at $data";
say "Data size is ", $size - $data
__END__
Dog
Cat
Bird

The program size includes the __END__ token and the newline after it. The rest belongs to the data:

File size is 164
Data starts at 151
Data size is 13

You can use DATA in other file things, including stat.

Read it twice

If you want to read the data twice, you can reset the file cursor. First, remember where DATA starts by calling tell before you read any lines. When you are ready to read it again, seek to that same position:

#!/usr/bin/perl
my $data_start = tell DATA;

print "---Outputting DATA\n", <DATA>, "---Done\n";

seek DATA, $data_start, 0;
print "---Outputting DATA\n", <DATA>, "---Done\n";
__END__
Dog
Cat
Bird

Using line numbers

#!/usr/bin/perl
my $data_start = tell DATA;

print "---Outputting DATA\n";
while( <DATA> ) {
	print "$. $_"
	}
print "---Done\n";

__END__
Dog
Cat
Bird

Now you see some line numbers, but those start counting from the first line under __DATA__:

---Outputting DATA
1 Dog
2 Cat
3 Bird
---Done

To get the real line numbers, you can figure out where the __END__ token is. This assumes that it’s not in the middle of documentation or in a string:

#!/usr/bin/perl
my $data_start = tell DATA;

my $end_line;
UNITCHECK {
	open my $fh, '<', $0;
	while( <$fh> ) { last if /\A__END__$/ }
	$end_line = $.
	}

print "---Outputting DATA\n";
while( <DATA> ) {
	$n = $end_line + $.;
	print "$n $_"
	}
print "---Done\n";

__END__
Dog
Cat
Bird

Now you see the offsets in the whole file and not the count after the __END__:

---Outputting DATA
19 Dog
20 Cat
21 Bird
---Done

There are some more vigorous methods in Can a Perl program know the line number where DATA begins? .

Multiple embedded files

This isn’t a DATA thing, but you can make several embedded files with Inline::Files:

#!/usr/bin/perl
use Inline::Files;

print "---Outputting dogs\n", <DOGS>, "---Done\n";
print "---Outputting cats\n", <CATS>, "---Done\n";
print "---Outputting birds\n", <BIRDS>, "---Done\n";
__DOGS__
Rin Tin Tin
Lassie
Ol' Yellar
__CATS__
Grumpy Cat
Garfield
Maru
Mr Bigglesworth
__BIRDS__
Woody Woodpecker
Roadrunner
Zazu
Sam the Eagle

Each of those get their own filehandles:

---Outputting dogs
Rin Tin Tin
Lassie
Ol' Yellar
---Done
---Outputting cats
Grumpy Cat
Garfield
Maru
Mr Bigglesworth
---Done
---Outputting birds
Woody Woodpecker
Roadrunner
Zazu
Sam the Eagle
---Done

Inline::Files has a problem because it overrides open. You have to use CORE::open to get to the real one:

use Inline::Files;

print "---Outputting dogs\n", <DOGS>, "---Done\n";
print "---Outputting cats\n", <CATS>, "---Done\n";
print "---Outputting birds\n", <BIRDS>, "---Done\n";

CORE::open my $fh, '<:utf8', '/etc/hosts' or die $!;
print "---Outputting hosts\n", <$fh>, "---Done\n";

Part 1

use strict;
use warnings;
##
# You are given an unsorted list of integers @N.
# Write a script to find out the smallest positive number missing.
##
sub least_missing{
    my(@numbers) = @_;
    @numbers = sort @numbers;
    for my $i ($numbers[0] .. $numbers[@numbers - 1]){
        my @a = grep { $_ == $i } @numbers;
        return $i if(!@a && $i > 0);
    }
    return undef;
}

MAIN:{
    my @N;
    @N = (5, 2, -2, 0);
    my $least_missing = least_missing(@N);
    print "The least mising number from (" .
        join(",", @N) . ") is $least_missing\n";
    @N = (1, 8, -1);
    $least_missing = least_missing(@N);
    print "The least mising number from (" .
        join(",", @N) . ") is $least_missing\n";
    @N = (2, 0, -1);
    $least_missing = least_missing(@N);
    print "The least mising number from (" .
        join(",", @N) . ") is $least_missing\n";
}

Sample Run

$ perl perl/ch-1.pl
The least mising number from (5,2,-2,0) is 1
The least mising number from (1,8,-1) is 2
The least mising number from (2,0,-1) is 1

Notes

The list is given in arbitrary order so the first thing to do is to sort it. Once in sorted order iterate from the least number to the highest, incrementing by one at each step. Perl makes this easy with the range (aka flip-flop) operator. Each each iteration see if the current number is from the original list or not and if not, then if it is the smallest positive number not yet seen, which really just means the first positive number not from the original list.

As I am writing this I realize that it’d make sense to use grep to remove all the negative numbers from the list before even bothering to sort them. If the list were presented as, say, 1 million negative numbers and then three positive ones why waste doing anything with all the negatives!

Part 2

use strict;
use warnings;
##
# You are given rankings of @N candidates.
# Write a script to find out the total candies needed for all candidates. 
# You are asked to follow the rules below:
#     a) You must given at least one candy to each candidate.
#     b) Candidate with higher ranking get more candies than their immediate
#        neighbors on either side.
##
sub count_candies{
    my(@candidates) = @_;
    my $candies = @candidates;
    for my $i (0 .. (@candidates - 1)){
        if($candidates[$i - 1]){
            $candies++ if $candidates[$i] > $candidates[$i - 1];
        }   
        if($candidates[$i + 1]){
            $candies++ if $candidates[$i] > $candidates[$i + 1];
        }
    }
    return $candies;
}


MAIN:{
    my @N;
    my $number_candies;
    @N = (1, 2, 2);
    $number_candies = count_candies(@N);
    print "The number of candies for (" .
        join(",", @N) . ") is $number_candies\n";
    @N = (1, 4, 3, 2);
    $number_candies = count_candies(@N);
    print "The number of candies for (" .
        join(",", @N) . ") is $number_candies\n";
}

Sample Run

$ perl perl/ch-2.pl
The number of candies for (1,2,2) is 4
The number of candies for (1,4,3,2) is 7

Notes

I don’t think there are any surprises in this approach. In fact, I could not think of a better way, in terms of efficiency, than this. Still, this is not exactly exciting code to read!

This month I interview Damian Conway, one of the Guardians of Perl. Damian is computer scientist and excellent communicator—his presentations and courses are widely popular around the world. He was the Adjunct Associate Professor in the Faculty of Information Technology at Melbourne’s Monash University between 2001 and 2010.

It was an honour to interview my idol. I enjoyed talking to him and I am sure you would have many “aha” moments. For example, Raku’s built-in grammar construct is inspired by the work of Damian’s Parse::RecDescent.

I also found out that Damian was the winner of Larry Wall Award for Practical Utility three years in a row from 1998 to 2000 for contributions to CPAN.

He also wrote some of the most popular books on Perl:

• Object Oriented Perl
• Perl Best Practices
• Perl Hacks

If you’d like me to interview you, or know someone you’d like me to interview, let me know. Take the same set of questions and send me your answers!

When and how did you get started with Perl?

In the mid-to-late 1990s, I was an academic at Australia’s largest university. At the time I was working on an unusually diverse range of research topics: educational technology, programming language design, software engineering, documentation systems, human-computer interaction, natural language generation, emergent systems, image morphing, human-computer interaction, API design, geometric modelling, the psychophysics of perception, nanoscale physical simulation, and parsing techniques. I liked to think I was a polymath, but the correct word is probably “dilettante”.

Anyway, I was implementing most of my software in C++, and I desperately needed some other language (any other language!) that would make text manipulation and generation, interface design, and programming language research a great deal easier. In particular, I was trying to solve problems in natural-language text generation, and so I searched the infant internet for a programming platform that would make that task, if not easy, then at least not painful. During that search I found Perl.

Within six months I had submitted two papers (on automating English inflexion, and on declarative command line interfaces) to the 2nd Perl Conference. Both were accepted and so I received funding from my institution to attend the conference. I don’t think I had been at the conference for more than half a day before I realized that I had finally found my tribe. And the rest is history.

What made you write Perl Best Practices?

Mostly it was two of the three Perl virtues: Impatience and Hubris. In the early 2000s I was becoming extremely impatient with the prevailing view outside the Perl community that Perl was a “write-only” language that produced code which was impossible to read, understand or maintain. And I was equally impatient with the prevailing view inside the Perl community that “More Than One Way To Do It” was a licence (or possibly a challenge) to do it in the most terse, tricky, and obscure ways possible.

Which led to hubris: the notion that I could find and teach a set of better habits for Perl coding; habits that would automatically produce code that was cleaner, more readable, more understandable, more robust, and more maintainable that the Perl code I was then seeing (well, to be honest, that I was then seeing mostly in my own terse, tricky, obscure module implementations).

So I sat down and reviewed every line of Perl code I had ever written, and a great many lines of Perl code that others had written, trying to find patterns of coding that consistently led either to excellent or to terrible code. Within six months I had over 250 such observations, and started teaching them in a two-day class. Soon after that, O’Reilly got in touch and suggested they might make a good book.

Do you follow your own suggestions in the book?

Yes. I feel it would be highly hypocritical of me not to.

However, I’ll admit that, at first, eating my own dog food was pure agony. A solid majority of my 250+ pieces of advice advised behaviours that I was not following myself at the time. I had to change just about everything from my curly-bracketing style (I was a confirmed BSD layout partisan, but PBP advocates K&R style), to my identifier-naming conventions (I originally didn’t have any at all, and was highly addicted to single-letter variables), to my commenting style (“Comments? My code is self-documenting!”), to the way I wrote OO Perl (usually without a framework, and always without proper encapsulation).

It took me months to replace all my bad habits with better ones, and even longer to download those new habits into my fingertips, so that they “just happened” whenever I was typing in code. But it’s been so very worthwhile. I can clearly see the difference between my pre-PBP code and the code I wrote after that time. I actually enjoy maintaining and improving the more recent code, whereas I find myself continually putting off any need to delve back into my older codebases.

In several cases (for example, Lingua::EN::Inflect or IO::Prompt) it was easier to completely rewrite the module (creating Lingua::EN::Inflexion and IO::Prompter respectively) rather than deal with that dreadful pre-PBP code.

Of course, there’s nothing intrinsically magical about the particular suggestions in the book. Indeed, in my current version of the class, somewhere around 20% of my advice has actually changed from what I wrote in PBP. The magic is in adopting and practising any set of consistent, well-thought-out, and productive coding habits.

What do you think about “Perl 7” currently being discussed widely?

I think it’s great to see Perl moving out from under the lingering ghost of “Perl 6”. And to see such a strong statement of positive forward motion, hopefully without too much of the attendant disruption of breaking vast swathes of existing code.

And, more importantly, I think it’s vital that the widely accepted boilerplate components of a modern Perl program (use strict, use warnings, declarative subroutine parameters, postfix dereferencing) should be turned on by default under new versions of Perl. And, of course, that some of the especially problematic vestigial components (indirect-object method calls, implicit hash-key concatenation, bareword filehandles) should be deprecated with extreme prejudice.

Of course, a plan this bold and this unanticipated will inevitably create anxiety and raise dissent. And not all those fears and disagreements will be misplaced. Even so, the very best thing about Perl 7 (whatever it ultimately proves to be) is that even just the idea of making a major version bump to usher in such fundamental changes is generating a huge amount of productive discussion and debate with the Perl community, and injecting a vast quantity of new energy, which must surely eventually lead to a better outcome, a better path forward for Perl.

Do you follow the development of “Cor”?

Yes, I’ve been following the Cor project with great interest for the past six months or so. I’ve occasionally also been consulting on the design of this proposed new mechanism, discussing particular issues and offering suggestions to Ovid on several aspects of the design. (Editor: See our earlier interview with Ovid)

Needless to say, I’m immensely excited by the prospect of having a genuinely declarative interface for class definitions added right into the core of the language! Even if that happy day is likely to be a few years away yet.

You have loads of handy modules published on CPAN. What are your top 5 contributions and why?

In no particular order:

Lingua::EN::Inflexion

This is the successor to my first ever public Perl module (Lingua::EN::Inflect), which was one of the two modules I presented at my first ever Perl conference, and which therefore in a sense paved the way for all the others. Both the original and the successor modules attempt to solve the very challenging problem of producing and recognizing correct word inflections in English (“cat”→”cats”, “box”→”boxes”, “ox”→”oxen”, “fish”→”fish”, “goose”→”geese”, “maximum”→”maxima”, “penny”→”pennies”, etc. etc.)

The reason I’m nominating the successor module instead of the original as one of my top five is that Lingua::EN::Inflexion is vastly more powerful, more reliable, and more maintainable, and reflects so much of what I have learned about the design and implementation of good APIs over the past two decades.

Parse::RecDescent

This was another early module, which provided what I like to think was the first truly Perlish grammatical parsing engine for the language. Prior to its release there had been ports of lex/yacc for Perl, but Parse::RecDescent allowed you to define grammars directly in your program, directly in Perl. It would also execute those grammars immediately, without a separate precompilation phase. The module has been very widely used over the past two decades in a huge number of commercial and academic projects. Finally, many of the ideas I pioneered in Parse::RecDescent eventually made their way into Raku’s built-in grammar construct, a contribution of which I’m immensely proud.

Quantum::Superpositions

In a similar vein, I have a huge fondness for my Quantum::Superpositions module. Not only because my presentation on that module has always been so hugely enjoyable to give, but also because that module led directly to the fundamental Raku concept of “junctions”, which I feel might be my only meaningful contribution to the field of computer science.

Regexp::Debugger

I use a great many regexes in my Perl solutions. When they go right, regexes are such powerful and efficient tools for solving complex data-processing problems, but when they go wrong, they’re an swirling abyss of confusion and frustration. That’s why I feel that Regexp::Debugger is probably the single most useful module I’ve ever written. It changes the task of debugging complex regexes from a multi-level, multi-day, multi-pain-killer nightmare into a simple, quick, “d’oh”-moment of sudden revelation. This module has saved me (and many other Perl users, I hope) literally thousands of hours of misery, simply by making our regex mistakes directly visible from right within the code itself.

PPR

Talking of complex regexes, I love the PPR module because it consists almost entirely of a single regex; a single 70000-character regex that can parse the vast majority of valid Perl documents. It’s such a mind-bogglingly simple and obvious idea; and yet it was so damn difficult to actually achieve. But worth every long hour of struggle. Having the ability to write a regex that matches a complete block of code, or a variable declaration, or a ‘for’ loop, or a ‘use’ statement, or any other Perl construct, opens up an entirely new universe of possibilities for validating, modifying, and refactoring Perl code. PPR is the basis for so many of the other CPAN modules of which I’m most proud, especially Keyword::Declare (which allows you to safely extend the Perl syntax using Perl itself), Dios (which brings most of Raku’s vastly superior OO model and syntax to Perl), and Code::ART (which provides Perl-refactoring tools for the Vim editor).

You visit Europe regularly and give lectures on Perl/Raku related topics. Do you have any memorable story to share?

I think my most memorable single experience in Europe was lecturing on Raku (or, rather, on Perl 6, as it was back then) at CERN in Geneva in 2015.

It was a pure joy to be in a room with so many brilliant people, talking with them about a project I have been so deeply committed to over the past two decades. And to watch that room full of brilliant people start to overcome their initial reluctance and begin to appreciate just how powerful this new language can be…that was a great moment.

Of course, it did require some chicanery to encourage them to put aside their scepticism and turn up in the first place. Specifically, I rearranged my standard talk so that I didn’t actually use the word “Perl” at all for nearly ninety minutes. In fact, I avoided naming the language in any way until the second-to-last slide.

And, indeed, it was that particular experience–seeing how easy it was to sidestep the widespread general prejudice against anything related to “Perl”–that made me so very keen to support the name change from “Perl 6” to “Raku” last year.

That visit to CERN was special to me for another reason too. Before my presentation, I had the privilege of being taken on a private tour of the facility. And though I didn’t manage to get down into the LHR itself (it was closed at the time for a refit), I did have my mind utterly blown by being allowed to wander around a building that seemed to be straight out of science fiction: the Antimatter Factory.

You briefly blogged about the weekly challenge, how did you find about it?

I learnt about the Weekly Challenge from the many blog posts from participants, which suddenly started popping up on my blogs.perl.org and reddit.com/r/perl+rakulang blog feeds. They seemed to be having so much fun that I couldn’t resist trying it myself.

And what was your experience with it?

I loved taking part in it. Not just for the individual challenges it offered me, but also for the opportunities it gave me to showcase some of the power, expressiveness, and convenience of solving those puzzles in Raku. And in doing so to reach out beyond the Perl and Raku communities to briefly capture the attention of the wider developer world.

It’s well known how much I adore teaching, and the Weekly Challenge gave me yet another opportunity and outlet to indulge that passion.

Where do you get the ideas for your next CPAN contributions?

I think most of my ideas come directly from my extremely low threshold of annoyance. Whenever I am in the middle of trying to solve a particular problem in Perl, and I can’t find a sufficiently simple way of doing so, I frequently get quite unreasonably frustrated. And my immediate response is to ask myself: “If I could make this work any way I wanted, how would I make it work?”

And then I remember that, being moderately competent in Perl, I can make it work in any way I want. So I do.

Mostly the actual ideas and solutions I implement come from a process I call “Design by Coding”. That is, when something is insufficiently easy, I just go ahead solve the problem the way I wish I could solve it, by writing the code that I wish I could write.

Of course, that doesn’t actually solve the problem, because this “gedankencode” won’t actually run. But then I merely need to spend a ridiculous and completely unjustifiable amount of time and energy making the hypothetical code work in real life. Typically, far more time and energy than it would have taken to just do it the hard way.

But the benefit is that, having expended all those hours on creating a better solution, thereafter I never have to do it the hard way again. And because I always put those solutions on CPAN, neither does anyone else.

At a deeper level, I think I’ve been able to come up with so many unusual ideas in my career because I spend a great deal of time “filling the well”. My wife is an artist, and she introduced me to this concept: that in order to be creative, you have to constantly refill your mind with a large number of novel and random facts, observations, and ideas. Creativity is then the process of finding new and unexpected connections between those ideas. The more ideas you have, the more links you can easily find, so the more creative you can be.

And that’s what I try to do. I try to read as diversely as I can, to give my creative processes the greatest possible range of building blocks and the greatest number of plausible (and implausible!) connections.

What would you suggest to someone starting Perl?

Three things:

1. Write bad code.
2. Study better code.
3. Read great books.

I think the fundamental mistake most beginners make, however, is that they try to do those three things in the wrong order: books first, other people’s code next, writing their own code last.

I don’t think there is any substitute for writing your own code from the very beginning. You need to grapple immediately with the challenges and puzzles of a new language. It’s the only way to actually improve as a programmer, especially in a language as eclectic and idiomatic as Perl.

Only after you have struggled to write Perl yourself, you can start to appreciate the better code that others may have written. And once you can begin to recognize good Perl code, you can start to read productively about Perl, to learn why that code is good.

Perl has so many great textbooks, but very few of them are completely sufficient if you’re not already at least a little familiar with the language itself. Without that prior struggle, I don’t think you can genuinely appreciate the “aha!” moments, the sudden opportunities for deep understanding, that a truly excellent textbook will offer you.

One other point I would make is that Perl almost suffers from a surfeit of excellent introductory textbooks…which can make it difficult for any particular beginner to find the one best suited to their own background, their personal learning style, and their individual goals.

Learning Perl might be perfect for one person, but someone else might do much better with Modern Perl. Some folks will find that a pragmatic use-case approach like Impatient Perl best meets their needs, whereas others would be far better served by a text that’s at almost the opposite extreme, like the CS-based theoretical framework of Elements of Programming with Perl.

The point is: if you’re looking for a book that will help you understand Perl, you should first try out at least a few of the many excellent alternatives available so you can find the one book that will help you understand Perl.

I noticed you are associated with Raku these days. Do you still use Perl?

Almost every day. I love Raku dearly, and I’m starting to use it more and more often in my daily coding, but when I need to get something critical implemented ASAP and then have it run as fast as possible on as wide a range of platforms as possible, Perl is still the tool I reach for almost every time.

For larger projects, Raku has some unbeatable advantages and in the next few years I expect its performance to become far more competitive, but for quick-and-dirty get-it-done-now tasks, Perl is still my preferred chainsaw.

And I suspect it always will be.

Part 1

You are given a positive number $N. Write a script to count the total number of set bits of the binary representations of all numbers from 1 to $N and return $total_count_set_bit % 1000000007.

Solution

Sample Run

$ perl perl/ch-1.pl
5 % 1000000007 = 5
4 % 1000000007 = 4

Notes

The approach here is to continuously shift bits off to the right, checking to see if the bit about to be shifted off is set or not. This is a pretty standard pattern and it looks pretty much the same in C++ and Prolog too!

Part 2

You are given an array of positive numbers @N. Write a script to represent it as Histogram Chart and find out how much water it can trap.

Sample Run

Notes

This is one of the more fun sorts of problems that come up in these challenges! It is somewhat similar to the “leader problem” from last week in that we are given an array of numbers and need to do a similar set of look ahead comparisons. Here we look ahead in the array to determine if what I call buckets exist. Whatever buckets are found are then used to compute the total volume as specified.

References

https://users.cs.cf.ac.uk/Dave.Marshall/PERL/node36.html

Creating open source code, documentation, books, videos is a lot of fun, but the creators still need to put food on the table. Some have full time jobs and do the public creation in their free time, others use time that otherwise would be sold as consulting hours.

In either case people appreciate when they have financial supporters.

This is the list of all the Perl-related authors and creators who are open to receive financial support for their contributions to the Perl Ecosystem.

Decide on some monthly budget you are willing to allocate to improve the Perl world (e.g. 10-20-30 USD/month) and the decide how do you split it up among the recipients.

The variable $^O also known as $OSNAME contains the the name of the operating system under which this copy of Perl was compiled.

During the quarantine I was able to find the good side of the home confination: I hadn’t enough time to read a book due to school’s tests, but for my luck, I had enough time for reading one or two Request For Comments (RFC) documents.

Since my first days studying computer security, the concept of “protocol” fascinated me. Maybe for their enormous diffusion in almost every computer system, our daily lives heavily depends from these processes. As I say “trust on machines but don’t trust humans”. The RFC approach reminds the open source philosophy, which has the same objective (give everyone the opportunity to learn new things through sharing) and the same propagation channel: the internet.

I find it too hard to search for these documents on the IETF website, so I made a fast and efficient script that permits me to download RFCs through a keyword and lets me decide which ones to read and which ones to ignore.

What are RFCs?

“Request For Comments” are one of the best online references for learning how networking really works at the protocol level. These are simply text documents that define every single standard that has been approved by the Internet Engineering Task Force. These documents are a good entry point for developing the study of protocol analysis: an emerging field that uses formal verification to find exploits.

Every RFC has a title and its associated number, for example the RFC 3607 is the “Chinese Lottery Cryptanalysis Revisited: The Internet as a Codebreaking Tool”.

My script uses Net::RFC::Search for the search, and Term::ANSIColor for the display—there will be a lot of text in the terminal and different coulrs can help!.

#!/usr/bin/perl
# Made by Edoardo Mantovani in 2020

use strict;
use warnings;

use File::Path qw(make_path);
use File::Spec::Functions;
use LWP::Protocol::https; # Net::RFC::Search needs this
use Net::RFC::Search;
use Term::ANSIColor;

my $folder = shift or die "./RFC <Folder> <Query>\n";
my $query  = shift or die "./RFC <Folder> <Query>\n";

make_path $folder;

my $rfc_interface = Net::RFC::Search->new();
my @rfc_nums = $rfc_interface->search_by_header( $query );

print "RFCs [@rfc_nums]\n";

foreach my $rfc_num ( @rfc_nums ) {
  my $local_file = catfile( $folder, $rfc_num ); # i.e final folder = /tmp/1110
  if ( $rfc_interface->get_by_index( $rfc_num, $local_file ) ) {
    print colored( "Downloaded: $rfc_num\n", "green" );
    view_rfc( $local_file );
  }
  else {
    print "Error with RFC $rfc_num\n";
  }
}

sub view_rfc {
  my( $file ) = @_;

  open my $fh, '<', $file or do {
    warn "Could not open file [$file]: $!\n";
    return;
  };

  while( <$fh> ) {
  	print;
  	next unless $. == 25;
    print colored("Do you want to save this RFC? [Y/N] ", "red");
    my $input = <STDIN>;
    chomp( $input );
	unlink $file if uc($input) eq "N";
    last;
  }
}

Running the program requires a directory name and a query. I create the directory if it doesn’t already exist:

$ ./rfc_search rfcs cookie

I display the first 25 lines of each matching RFC and prompt to save it.

How it works

The search_by_header returns the RFC numbers for each of the documents that has the query word:

my @rfc_nums = $rfc_interface->search_by_header( $query );

I then fetch each document by specifying its number and local filename. I do this even if I won’t keep it because I don’t know what’s in it yet:

$rfc_interface->get_by_index( $rfc_num, $local_file );

For the last part, I look at the first 25 lines of the file. I have a prompt that asks me if I want to save it. If I enter “N”, I delete that file. I color the prompt so I can see it against the RFC text:

sub view_rfc {
  my( $file ) = @_;

  open my $fh, '<', $file or do {
    warn "Could not open file [$file]: $!\n";
    return;
  };

  while( <$fh> ) {
  	print;
  	next unless $. == 25;
    print colored("Do you want to save this RFC? [Y/N] ", "red");
    my $input = <STDIN>;
    chomp( $input );
    rmtree(@_) if uc($input) eq "N";
    last;
  }
}

Other uses and suggestions

I made this simple script mainly for save time and have a better interface for interacting with the RFC site, I want to recommend to see the TODO part in the Net::RFC::Search page. For a better script, it would be cool to implement curl and lynx to retrieve cancelled RFC’s. Of course the script can be implemented with custom functions but this is script is only for educational purposes.

resources - Request for Comments: User Stories and Scenarios - Wikipedia’s RFC entry

The $" variable, also called the $LIST_SEPARATOR indicates what goes between array elements when they are intepolated in a string.