All,
company is 'tightening the screws' so to speak, and only allowing software systems
available for download if they are secured in some way.
So - I've found myself in a position I'd otherwise not want to be in - trying to sell
the ISS folks here on the security of CPAN.
I've said that the modules available on CPAN are verified by a MD5 checksumming mechanism,
but my guess is that they are going to say that that is not enough. My guess is that they
are going to want SHA-256 or SHA-512 in order to download and use perl modules from CPAN.
Are there any plans to allow for this greater level of security on CPAN?
And If not, would it help if I asked really nice for it to be implemented? ;-)
Ed

Ed Peschko <esp5 (AT) pge (DOT) comwrites:

I've said that the modules available on CPAN are verified by a MD5
checksumming mechanism, but my guess is that they are going to say
that that is not enough. My guess is that they are going to want
SHA-256 or SHA-512 in order to download and use perl modules from
CPAN.

What kind of security are they trying to achieve?

CPAN is an unorganised anarchy. The checksum only guarantees that your
downloaded file is identical to the one on CPAN, but there's no
guarantee whatsoever that the file on CPAN is sane. There's no global
authority that takes responsibility for anything downloaded from CPAN.

Are there any plans to allow for this greater level of security on
CPAN?

I don;t think so, although it may not be that hard to add. It's only
that it will not increase the security at all
-- Johan

Tue, Jul 04, 2006 at 10:16:09AM +0200, Johan Vromans wrote:
Ed Peschko <esp5 (AT) pge (DOT) comwrites:

I've said that the modules available on CPAN are verified by a MD5
checksumming mechanism, but my guess is that they are going to say
that that is not enough. My guess is that they are going to want
SHA-256 or SHA-512 in order to download and use perl modules from
CPAN.

What kind of security are they trying to achieve?

CPAN is an unorganised anarchy. The checksum only guarantees that your
downloaded file is identical to the one on CPAN, but there's no
guarantee whatsoever that the file on CPAN is sane. There's no global
authority that takes responsibility for anything downloaded from CPAN.

Are there any plans to allow for this greater level of security on
CPAN?

I don;t think so, although it may not be that hard to add. It's only
that it will not increase the security at all

Yes, I'm aware that this is all pretty much pointless - if someone want to
post an exploit, MD5 and and SHA-256 codes aren't going to make one jot of difference.

That being said, I trust my - and the communities' - code auditing (and system monitoring)
ability plus the track record that in about 8 years of downloading CPAN modules I've yet to
see one security issue from downloading ANYTHING.

However, these arguments are not very persuasive in some quarters. IM, its just that
people in authority positions in companies want to see the right acronyms. And I can't
hesitate to say that I prefer the 'good old trustful days' to all of this (somewhat needless)
paranoia.

Ed

Wed, 2006-07-05 at 00:40 -0700, Ed Peschko wrote:
Tue, Jul 04, 2006 at 10:16:09AM +0200, Johan Vromans wrote:
Ed Peschko <esp5 (AT) pge (DOT) comwrites:

I've said that the modules available on CPAN are verified by a MD5
checksumming mechanism, but my guess is that they are going to say
that that is not enough. My guess is that they are going to want
SHA-256 or SHA-512 in order to download and use perl modules from
CPAN.

What kind of security are they trying to achieve?

CPAN is an unorganised anarchy. The checksum only guarantees that your
downloaded file is identical to the one on CPAN, but there's no
guarantee whatsoever that the file on CPAN is sane. There's no global
authority that takes responsibility for anything downloaded from CPAN.

Are there any plans to allow for this greater level of security on
CPAN?

I don;t think so, although it may not be that hard to add. It's only
that it will not increase the security at all

Yes, I'm aware that this is all pretty much pointless - if someone want to
post an exploit, MD5 and and SHA-256 codes aren't going to make one jot of difference.

It could. It can show you that a mirror has been altered. (And only if
you compare across mirrors.)

That being said, I trust my - and the communities' - code auditing (and system monitoring)
ability plus the track record that in about 8 years of downloading CPAN modules I've yet to
see one security issue from downloading ANYTHING.

CPAN has been lucky. Trojaned tarballs have been placed on a number of
open source project archives over the last few years. Many were caught
because of those obsessive compulsive types who are paying attention to
what gets uploaded. Many were caught by digital signature tests. (At
least one was caught by Gentoo's automatic signature testing on their
build tool.)

These were not small projects either. Attempts have been made on
Apache.org and other high profile targets. (Someone even tried to insert
a trojaned patch into the Linux kernel bitkeeper repository. They were
caught within minutes.)

However, these arguments are not very persuasive in some quarters. IM, its just that
people in authority positions in companies want to see the right acronyms. And I can't
hesitate to say that I prefer the 'good old trustful days' to all of this (somewhat needless)
paranoia.

There *is* something to worry about. Unfortunately those in authority
want processes that do little or nothing to "secure" anything. They
want buzzwords, not actual effective methods. (Mainly because far too
many people who are tasked with security have absolutely no
understanding of it. Anything they do not understand is a threat. Most
corporate security people would not know a "threat model" from a
"supermodel".)

In my opinion, ALL tarballs should be signed by GPG keys of known
individuals.

If the signature module is the preferred signing method, then it should
be in core. (Source tarballs should be signed with detached
signatures.)

There needs to be an authentication method to know that whoever signed
the module or tarball was authorized to do so. (A maintainers list with
key ids would be a start.)

There need to be tools to periodically check CPAN mirrors for altered or
unsigned tarballs.

[I am probably missing something that I will remember when I hit
'send'.]

Done with the right tools and the right process, it is pretty painless.

Getting hacked and having a trojaned version distributed to n+1 people
is not.

During next YAPC::EU, Sam Vilain and I will present the design of a
possible follow-up on CPAN: CPAN6. The user side is still the same,
but internally it is very different (not only the security issues).
I am looking for sponsors to implement it, but we first need to discuss
this with the Perl community. Its use is not limited to Perl.

Mark

Mark

The single biggest fault I've seen so far with respect to every proposal
for CPAN6 or the 6PAN or whatever we call it is a serious lack of
consultation.

If anyone is proposing a design for a system that everyone is going to
have to live with the foibles of for the next two decades, I would
expect to see said design circulated for everybody involved in the
current CPAN (in the sense of development and management).

That list would probably include, Andreas, Brian, me, P5P, the various
platform groups such as Win32 and VMS and S/390, the developers of
EU:MM, M:B, M:I, CPANPLUS, and so on (Ken, Schwern, Jos, Audrey, Ingy et
al), and various people from the downstream distributions.

And I don't mean some of these people, I mean ALL of them.

At the very least all these people need to be challenging your design,
and you need to be able to stand up to any criticism, and where
necesary, adapt the design to take into account their use cases.

Can the design handle 50,000 authors, 100,000 packages, and 200 million
lines of code? Can you prove it? Does it fit with Perl 6's ideas of how
the universe works? Can you prove that? And so on

I'm not suggesting you aren't planning on doing it already, but while a
presenting at a conference might be an appropriate launching pad for
your concept, it should be at best an initial introduction the the
expectation that it will change.

Because the CPAN is large enough and all-pervasive enough that I doubt
any of us are qualified to design a sequel, and at some level it needs
to become a large group project (without falling into second system etc
problems).

Adam K

Mark wrote:
* Adam Kennedy (adam (AT) phase-n (DOT) com) [060708 03:47]:
During next YAPC::EU, Sam Vilain and I will present the design of a
possible follow-up on CPAN: CPAN6.

>The single biggest fault I've seen so far with respect to every proposal
>for CPAN6 or the 6PAN or whatever we call it is a serious lack of
>consultation.

For one, I have experienced the Perl community long enough to know
how it works. For two: why do you think there is a lack of consultation?
The last two years I am walking around on European YAPCs to get people
interested in a more serious redevelopment of CPAN, but that did not
trigger anyone. minor fixes where suggested, IM those are
unsufficient.

>That list would probably include, Andreas,
>And I don't mean some of these people, I mean ALL of them.

The coming YAPC::EU would be a nice moment to start some working groups.
I hope for interesting BoFs.

Conferences are terrible places to start working groups, or do anything
more than gossip and muse, because no matter what the conference you are
only going to, you're only seeing a small fraction of the people
involved on any given topic.

Even the YAPC::NA hackfest, which had the authors of something like
10%-20% of CPAN in one room, still wasn't sufficient to do more than
have full bandwidth conversations to clear up things that had previously
been aired in other forums.

Progress was made in a wide range of areas, but apart from a few
concentrated efforts like Win32 or Perl 6, almost all the ideas get
reflected in various mailing lists (take the current TAP stuff in the QA
list for example).

>Can the design handle 50,000 authors, 100,000 packages, and 200 million
>lines of code? Can you prove it? Does it fit with Perl 6's ideas of how
>the universe works? Can you prove that? And so on

These numbers are quite much smaller than I expect, because CPAN6 is not
restricted to Perl5.

Quite possibly, but one order of magnitude is a good minimum spec.

>Because the CPAN is large enough and all-pervasive enough that I doubt
>any of us are qualified to design a sequel, and at some level it needs
>to become a large group project (without falling into second system etc
>problems).

The main reason why I kept low profile, it that most people in the Perl
community see CPAN as one homogeous process, where it deserves better.
At least, you should seperate clearly the infrastructure (mirrors
etc), administration (pause), and installation tools. after
that separation is clear to most people, the next step can be made.
The installation tools are hardly affected.

It's one of the nicest features of CPAN that people on the outside see
it as one system. But most of the people involved in building and
maintaining it understand what the real situation is.

They are the ones you should be talking to if you have ideas about
building a new CPAN. They'll understand exactly what you are talking
about, because they'll have similar ideas to yours about how to make
things better.

If you had read-up on my background, you may have phrased your e-mail
differently. Its usually more constructive to listen to new ideas first,
before attacking the messenger. And it is usually better to think about
new ideas thorowly, before presenting them to a larger group. Don't
worry, you get your chance. Hope to see you in Birmingham (or after that
on a cpan6 mailinglist)

Well, I'm afraid I can only fit one round the world trip into my
schedule and budget this year, so I won't be there. But I look forward
to seeing a repeat of your presentation at YAPC::AU. The deadline for
talks closes in about a week. :)

But my point is really that Conferences Don't Count. At any given
conference you address only the smallest portion of the community, most
of which you will never know about and never meet.

The best feature about this year's YAPC::NA will be that for the first
time we can broadcast these same talks in high quality to the WHLE of
the community.

And I have nothing against the messenger R the message. It's just that
even without knowing you beyond the dozen modules you've release, or
your ideas at all, I'm fairly willing to assert that it's not good
enough, because I think it's intrinsically the nature of CPAN that it is
so large, with so many components all interlinked, that no single person
or group is going to be able to design a suitably good replacement, and
that it's going to take input from most or all of the existing
participants (the list I mentioned) AND input from new contributors like
yourself, in order to come to a suitable result.

That said, I'm quite interested in your ideas. It's just that I'd prefer
to see them now, rather than wait until after they've been formally
presented.

I'd rather not only for you to have thought about them, but that you
bounce the ideas of many other individuals (NT in public) for review,
and THEN present to larger groups.

Adam K

Johan Vromans wrote:
Adam Kennedy <adam (AT) phase-n (DOT) comwrites:

several technical and practical issues, like:

>Can the design handle 50,000 authors, 100,000 packages, and 200
>million lines of code? Can you prove it?

I think details like this are unimportant in this stage. First more
fundamental issues have to be dealt with, like: What is CPAN6 supposed
to do? For what people? For what purpose?

We need a vision first.

The term vision might be a bit dangerous though.

Clearly we do need to have some idea of what the CPAN should do, but it
can be easily to get pulled into some exciting future.

Personally, I think that the CPAN is much more a boring civil
engineering effort.

Clearly it has a set of purposes, a scope, a scale, and operates under a
set of constraints and conditions in a defined set of environments.

But I think one of the biggest concerns is that the CPAN is something
very special, and most of the people that know how it works are paranoid
about screwing up and ruining it for everyone.

I'm all for discussions on what CPAN should be, but on this matter I'm
vary of grand visions. LSD might be great for musicians, but nobody
wants the guy building a bridge to be high.

With the caveat that any ideas about what it needs to do be firmly
anchored in reality though, I do agree with you.

Adam K

Adam Kennedy <cpan (AT) ali (DOT) aswrites:

Clearly we do need to have some idea of what the CPAN should do, but
it can be easily to get pulled into some exciting future.
Personally, I think that the CPAN is much more a boring civil
engineering effort.

This _is_ a vision, and a rather clear one.

[] but on this matter I'm vary of grand visions. LSD might be
great for musicians, but nobody wants the guy building a bridge to
be high.

(Assuming you meant "weary") Fortunately my ideas of a vision are a
bit more realistic. Sometimes too realistic
-- Johan

Adam Kennedy <adam (AT) phase-n (DOT) comwrites:

several technical and practical issues, like:

Can the design handle 50,000 authors, 100,000 packages, and 200
million lines of code? Can you prove it?

I think details like this are unimportant in this stage. First more
fundamental issues have to be dealt with, like: What is CPAN6 supposed
to do? For what people? For what purpose?

We need a vision first.
-- Johan

Mark <mark (AT) overmeer (DOT) netwrites:

The last two years I am walking around on European YAPCs to get
people interested in a more serious redevelopment of CPAN, but that
did not trigger anyone. minor fixes where suggested, IM those
are unsufficient.

For a start, noone seems to have a clear vision of what CPAN6 should
achieve. Most, if not all, proposals I've seen so far concentrate on
solving problems with the current CPAN.

I think Mark is right that we need something different for CPAN6
(CUAN?). But it does not start with a design, but with a vision. After
many discussions and debates, being active in the Perl community since
Perl 1, I do have some ideas, but I would not dare to call it a
vision. I've heard many ideas from other people, but no vision.

So, before starting a new round of debates, I'd like to hear the
vision behind any new CPAN plans. If the first step is the
presentation of a design, then it's just CPAN5 fixing time again.
-- Johan