PERLTODO

NAME

DESCRIPTION

Whilst patches to make the list shorter are most welcome, ideas to add to the list are also encouraged. Check the perl5-porters archives for past ideas, and any discussion about them. One set of archives may be found at:

    http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/

What can we offer you in return? Fame, fortune, and everlasting glory? Maybe not, but if your patch is incorporated, then we'll add your name to the AUTHORS file, which ships in the official distribution. How many other programming languages offer you 1 line of immortality?  

Tasks that only need Perl knowledge

Remove duplication of test setup.

merge common code in installperl and installman

common test code for timed bail out

POD -> HTML conversion in the core still sucks

The addition of "Pod::Simple" and its related modules may make this task easier to complete.  

merge checkpods and podchecker

perlmodlib.PL rewrite

Parallel testing

The core regression test suite is getting ever more comprehensive, which has the side effect that it takes longer to run. This isn't so good. Investigate whether it would be feasible to give the harness script the option of running sets of tests in parallel. This would be useful for tests in t/op/*.t and t/uni/*.t and maybe some sets of tests in lib/.

Questions to answer

1.

How does screen layout work when you're running more than one test?

2.

How does the caller of test specify how many tests to run in parallel?

3.

How do setup/teardown tests identify themselves?

Pugs already does parallel testing - can their approach be re-used?  

Make Schwern poorer

Improve the coverage of the core tests

test B

Deparse inlined constants

    use constant PI => 4;
    warn PI

will currently deparse as

    use constant ('PI', 4);
    warn 4;

because the tokenizer inlines the value of the constant subroutine "PI". This allows various compile time optimisations, such as constant folding and dead code elimination. Where these haven't happened (such as the example above) it ought be possible to make B::Deparse work out the name of the original constant, because just enough information survives in the symbol table to do this. Specifically, the same scalar is used for the constant in the optree as is used for the constant subroutine, so by iterating over all symbol tables and generating a mapping of SV address to constant name, it would be possible to provide B::Deparse with this functionality.  

A decent benchmark

fix tainting bugs

Dual life everything

To make a minimal perl distribution, it's useful to look at t/lib/commonsense.t.  

Improving threads::shared

POSIX memory footprint

embed.pl/makedef.pl

use strict; and AutoLoad

    package Whack;
    use AutoLoader 'AUTOLOAD';
    use strict;
    1;
    __END__
    sub bloop {
        print join (' ', No, strict, here), "!\n";
    }

then "use strict;" isn't in force within the autoloaded subroutines. It would be more consistent (and less surprising) to arrange for all lexical pragmas in force at the __END__ block to be in force within each autoloaded subroutine.

There's a similar problem with SelfLoader.  

Tasks that need a little sysadmin-type knowledge

make HTML install work

1.

Checking that cross linking between various parts of the documentation works. In particular that links work between the modules (files with POD in lib/) and the core documentation (files in pod/)

2.

Work out how to split "perlfunc" into chunks, preferably one per function group, preferably with general case code that could be used elsewhere. Challenges here are correctly identifying the groups of functions that go together, and making the right named external cross-links point to the right page. Things to be aware of are "-X", groups such as "getpwnam" to "endservent", two or more "=items" giving the different parameter lists, such as

    =item substr EXPR,OFFSET,LENGTH,REPLACEMENT
    =item substr EXPR,OFFSET,LENGTH
    =item substr EXPR,OFFSET

and different parameter lists having different meanings. (eg "select")

compressed man pages

Add a code coverage target to the Makefile

*

do a normal "Configure", but include Devel::Cover as a module to install (see INSTALL for how to do this)

*

    make perl

*

    cd t; HARNESS_PERL_SWITCHES=-MDevel::Cover ./perl -I../lib harness

*

Process the resulting Devel::Cover database

This just give you the coverage of the .pms. To also get the C level coverage you need to

*

Additionally tell "Configure" to use the appropriate C compiler flags for "gcov"

*

    make perl.gcov

(instead of "make perl")

*

After running the tests run "gcov" to generate all the .gcov files. (Including down in the subdirectories of ext/

*

(From the top level perl directory) run "gcov2perl" on all the ".gcov" files to get their stats into the cover_db directory.

*

Then process the Devel::Cover database

It would be good to add a single switch to "Configure" to specify that you wanted to perform perl level coverage, and another to specify C level coverage, and have "Configure" and the Makefile do all the right things automatically.  

Make Config.pm cope with differences between built and installed perl

It would be good to find a way teach "Config.pm" about the installation setup, possibly involving probing at install time or later, so that the %Config in a binary distribution better describes the installed machine, when the installed machine differs from the build machine in some significant way.  

linker specification files

Cross-compile support

This could be done little differently. Namely "miniperl" should be built for HOST and then full "perl" with extensions should be compiled for TARGET. This, however, might require extra trickery for %Config: we have one config first for HOST and then another for TARGET. Tools like MakeMaker will be mightily confused. Having around two different types of executables and libraries (HOST and TARGET) makes life interesting for Makefiles and shell (and Perl) scripts. There is $Config{run}, normally empty, which can be used as an execution wrapper. Also note that in some cross-compilation/execution environments the HOST and the TARGET do not see the same filesystem(s), the $Config{run} may need to do some file/directory copying back and forth.  

roffitall

Tasks that need a little C knowledge

Exterminate PL_na!

Modernize the order of directories in @INC

-Duse32bit*

Make it clear from -v if this is the exact official release

It would be useful to find an elegant way to have the ``this is an interim maintenance release'' or ``this is a release candidate'' in the terse -v output, and have it so that it's easy for the pumpking to remove this just as the release tarball is rolled up. This way the version pulled out of rsync would always say ``I'm a development release'' and it would be safe to bump the reported minor version as soon as a release ships, which would aid perl developers.

This task is really about thinking of an elegant way to arrange the C source such that it's trivial for the Pumpking to flag ``this is an official release'' when making a tarball, yet leave the default source saying ``I'm not the official release''.  

Profile Perl - am I hot or not?

As part of this, the idea of pp_hot.c is that it contains the hot ops, the ops that are most commonly used. The idea is that by grouping them, their object code will be adjacent in the executable, so they have a greater chance of already being in the CPU cache (or swapped in) due to being near another op already in use.

Except that it's not clear if these really are the most commonly used ops. So as part of exercising your skills with coverage and profiling tools you might want to determine what ops really are the most commonly used. And in turn suggest evictions and promotions to achieve a better pp_hot.c.  

Allocate OPs from arenas

Note that Configuring perl with "-Accflags=-DPL_OP_SLAB_ALLOC" will use Perl_Slab_alloc() to pack optrees into a contiguous block, which is probably superior to the use of OP arenas, esp. from a cache locality standpoint. See ``Profile Perl - am I hot or not?''.  

Improve win32/wince.c

Use secure CRT functions when building with VC8 on Win32

    FILE* f = fopen(__FILE__, "r");

one should now write

    FILE* f;
    errno_t err = fopen_s(&f, __FILE__, "r");

Currently, the warnings about these deprecations have been disabled by adding -D_CRT_SECURE_NO_DEPRECATE to the CFLAGS. It would be nice to remove that warning suppressant and actually make use of the new secure CRT functions.

There is also a similar issue with POSIX CRT function names like fileno having been deprecated in favour of ISO C++ conformant names like _fileno. These warnings are also currently suppressed by adding -D_CRT_NONSTDC_NO_DEPRECATE. It might be nice to do as Microsoft suggest here too, although, unlike the secure functions issue, there is presumably little or no benefit in this case.  

strcat(), strcpy(), strncat(), strncpy(), sprintf(), vsprintf()

  nm libperl.a | ./miniperl -alne '$o = $F[0] if /:$/; print "$o $F[1]" if $F[0] eq "U" && $F[1] =~ /^(?:strn?c(?:at|py)|v?sprintf|gets)$/'

Note, of course, that this will only tell whether your platform is using those naughty interfaces.  

-D_FORTIFY_SOURCE=2, -fstack-protector

Tasks that need a knowledge of XS

autovivification

This task is incremental - even a little bit of work on it will help.  

Unicode in Filenames

Known combinations that have some level of understanding include Microsoft NTFS, Apple HFS+ (In Mac OS 9 and X) and Apple UFS (in Mac OS X), NFS v4 is rumored to be Unicode, and of course Plan 9. How to create Unicode filenames, what forms of Unicode are accepted and used (UCS-2, UTF-16, UTF-8), what (if any) is the normalization form used, and so on, varies. Finding the right level of interfacing to Perl requires some thought. Remember that an OS does not implicate a filesystem.

(The Windows -C command flag ``wide API support'' has been at least temporarily retired in 5.8.1, and the -C has been repurposed, see perlrun.)

Most probably the right way to do this would be this: ``Virtualize operating system access''.  

Unicode in %ENV

Unicode and glob()

Unicode and lc/uc operators

use less 'memory'

This task is incremental - even a little bit of work on it will help.  

Re-implement :unique in a way that is actually thread-safe

Make tainting consistent

readpipe(LIST)

Audit the code for destruction ordering assumptions

    /* Need to check SvMAGICAL, as during global destruction it may be that
       AvARYLEN(av) has been freed before av, and hence the SvANY() pointer
       is now part of the linked list of SV heads, rather than pointing to
       the original body.  */
    /* FIXME - audit the code for other bugs like this one.  */

adding the "SvMAGICAL" check to

    if (AvARYLEN(av) && SvMAGICAL(AvARYLEN(av))) {
        MAGIC *mg = mg_find (AvARYLEN(av), PERL_MAGIC_arylen);

Go through the core and look for similar assumptions that SVs have particular types, as all bets are off during global destruction.  

Extend PerlIO and PerlIO::Scalar

Similarly the PerlIO vtable doesn't know about formats (write()), or about stat(), or chmod()/chown(), utime(), or flock().

(For PerlIO::Scalar it's hard to see what e.g. mode bits or ownership would mean.)

PerlIO doesn't do directories or symlinks, either: mkdir(), rmdir(), opendir(), closedir(), seekdir(), rewinddir(), glob(); symlink(), readlink().

See also ``Virtualize operating system access''.  

-C on the #! line

Propagate const outwards from Perl_moreswitches()

Duplicate logic in S_method_common() and Perl_gv_fetchmethod_autoload()

        /* This code tries to figure out just what went wrong with
           gv_fetchmethod.  It therefore needs to duplicate a lot of
           the internals of that function.  We can't move it inside
           Perl_gv_fetchmethod_autoload(), however, since that would
           cause UNIVERSAL->can("NoSuchPackage::foo") to croak, and we
           don't want that.
        */

If "Perl_gv_fetchmethod_autoload" gets rewritten to take (more) flag bits, then it ought to be possible to move the logic from "S_method_common" to the ``right'' place. When making this change it would probably be good to also pass in at least the method name length, if not also pre-computed hash values when known. (I'm contemplating a plan to pre-compute hash values for common fixed strings such as "ISA" and pass them in to functions.)  

Organize error messages

This would enable all sorts of things: easier translation/localization of the messages (though please do keep in mind the caveats of Locale::Maketext about too straightforward approaches to translation), filtering by severity, and instead of grepping for a particular error message one could look for a stable error id. (Of course, changing the error messages by default would break all the existing software depending on some particular error message...)

This kind of functionality is known as message catalogs. Look for inspiration for example in the catgets() system, possibly even use it if available--- but only if available, all platforms will not have catgets().

For the really pure at heart, consider extending this item to cover also the warning messages (see perllexwarn, "warnings.pl").  

Tasks that need a knowledge of the interpreter

UTF-8 revamp

Properly Unicode safe tokeniser and pads.

state variable initialization in list context

    state ($a, $b) = foo();

In Perl 6, "state ($a) = foo();" and "(state $a) = foo();" have different semantics, which is tricky to implement in Perl 5 as currently they produce the same opcode trees. The Perl 6 design is firm, so it would be good to implement the necessary code in Perl 5. There are comments in "Perl_newASSIGNOP()" that show the code paths taken by various assignment constructions involving state variables.  

Implement $value ~~ 0 .. $range

A does() built-in

Tied filehandles and write() don't mix

Attach/detach debugger from running program

Optimize away empty destructors

LVALUE functions for lists

LVALUE functions in the debugger

regexp optimiser optional

delete &function

/w regex modifier

    do { local $"='|'; /\b(?:P)\b/ }

See <http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2007-01/msg00400.html> for the discussion.  

optional optimizer

You WANT *how* many

lexical aliases

entersub XS vs Perl

Self-ties

Optimize away @_

The yada yada yada operators

The ... operator is the ``yada, yada, yada'' list operator, which is used as the body in function prototypes. It complains bitterly (by calling fail) if it is ever executed. Variant ??? calls warn, and !!! calls die.

Those would be nice to add to Perl 5. That could be done without new ops.  

Virtualize operating system access

This has actually already been implemented (but only for Win32), take a look at iperlsys.h and win32/perlhost.h. While all Win32 variants go through a set of ``vtables'' for operating system access, non-Win32 systems currently go straight for the POSIX/UNIX-style system/library call. Similar system as for Win32 should be implemented for all platforms. The existing Win32 implementation probably does not need to survive alongside this proposed new implementation, the approaches could be merged.

What would this give us? One often-asked-for feature this would enable is using Unicode for filenames, and other ``names'' like %ENV, usernames, hostnames, and so forth. (See ``When Unicode Does Not Happen'' in perlunicode.)

But this kind of virtualization would also allow for things like virtual filesystems, virtual networks, and ``sandboxes'' (though as long as dynamic loading of random object code is allowed, not very safe sandboxes since external code of course know not of Perl's vtables). An example of a smaller ``sandbox'' is that this feature can be used to implement per-thread working directories: Win32 already does this.

See also ``Extend PerlIO and PerlIO::Scalar''.  

Investigate PADTMP hash pessimisation

Big projects

make ithreads more robust

This task is incremental - even a little bit of work on it will help, and will be greatly appreciated.

One bit would be to write the missing code in sv.c:Perl_dirp_dup.

Fix Perl_sv_dup, et al so that threads can return objects.  

iCOW

(?{...}) closures in regexps

A re-entrant regexp engine

Add class set operations to regexp engine

demerphq has this on his todo list, but right at the bottom.