Saturday, April 12, 2008

pkgcore the victor

pkgcore is my pick for the next portage and codebase for emerge-ng. I admit I haven't let paludis have a chance (yet).

Why haven't I let paludis have it's chance?

The reasons are quite simple. Everytime I have tried paludis prior to this it has required extra configuration (meaning it doesn't work out of the box). I believe this was fixed recently. However,


paludis@1208029212: [WARNING] Use of Portage configuration files will lead to sub-optimal performance and loss of functionality. Full support for Portage configuration formats is not guaranteed; issues should be reported via trac. You are strongly encouraged to migrate to a Paludis configuration.
paludis@1208029212: [WARNING] Use of Portage configuration files will lead to sub-optimal performance and loss of functionality. Full support for Portage configuration formats is not guaranteed; issues should be reported via trac. You are strongly encouraged to migrate to a Paludis configuration.
* Done cleaning write cache for ebuild format repositories
paludis@1208029212: [WARNING] Use of Portage configuration files will lead to sub-optimal performance and loss of functionality. Full support for Portage configuration formats is not guaranteed; issues should be reported via trac. You are strongly encouraged to migrate to a Paludis configuration.



This is the end of a paludis --sync. Annoying. Paludis also has it's alpha releases marked as ~arch, this is a no, no. I also found the people in #paludis on irc to be rude and unhelpful. I was annoyed at the time (so being rude myself) as gentoo kde-svn overlay was going to require paludis which means I was going to be forced to use it. Regardless I've never been met with lot's of friendliness or help in #paludis. They are an RTFM crowd. I have been told to use paludis several times without even asking about it (as if it is the solution to all problems). So is pkgcore better? I can't say without a doubt that it is.

Why is pkgcore the victor?

For starter's it's only the victor for me. paludis is a more mature product at this point and has a stronger following of fanatic's/zealot's.

1.) pkgcore it more similar to portage.

pkgcore was originally intended to be portage 3. This leads to a large amount of similarities and backwards compatibility.

2.) pkgcore devs are really friendly. Sometimes patience is a virtue getting an answer on #pkgcore but most of the times I get one. This allowed me to quickly learn what I could and could not (yet) do with pkgcore, including finding a bug within a week.

3.) pkgcore just works. I installed pkgcore and with a quick look at the site was quickly able to start doing work with it.

4.) pkgcore installs faster. I think it took like an hour to build paludis on this machine (maybe more) pkgcore took 2 minutes tops.

5.) pkgcore uses C. For the parts of portage that were really slow they re-wrote them in C using cpython. using both python and c leads to faster development and speed where needed.

How do I use pkgcore
real quick

pmaint sync

this is the biggest adaption from portage, imho and I'm not sure why it's not --sync or pmerge --sync (in some ways it's more similar to portage 2.0 syntax). This will not only sync your main gentoo tree but any repositories you've intalled with layman. as of yet you still have to install them via layman, but this may change in the future.

pmerge packagename

I told you this was easy. Most of the options I use work. The biggest that doesn't is --verbose, this seems to be default now (with --ask) so an option is no longer needed. updating world

pmerge -auDs world

an -s is now needed to work with things like system and world. It's for package set. My one complaint here is the --newuse doesn't currently work with sets. This is a bug and will be fixed in the future.

Is anything wrong with pkgcore in your eyes?

well aside from the plethora of missing features (they'll get done, it's just a time thing). Documentation... I usually learn stuff by reading web documentation (or books) until I'm comfortable enough reading man pages. The web documentation for pkgcore needs a lot of work. I may actually volunteer to help with this. I haven't decided yet, and it depends partially on if they want to answer all my questions as I write the docs.

It'd be nice if at some point pmerge could become emerge etc... (meaning reprise it's roll as portage 3).

for more information on pkgcore go to pkgcore.org

15 comments:

Anonymous said...

"For starter's it's only the victor for me. paludis is a more mature product at this point and has a stronger following of fanatic's/zealot's."

It's about the community, the users and the people - not about being right?

I guess it could be seen as a quality that you're bold enough to knowingly base your fork on less mature products. I'm not sure how you're going to explain the benefits of less mature code to potential users though.

xenoterracide said...

"It's about the community, the users and the people - not about being right?"

for varying definitions of 'right'. I do not believe paludis to be right about anything. Just my opinion.

"guess it could be seen as a quality that you're bold enough to knowingly base your fork on less mature products. I'm not sure how you're going to explain the benefits of less mature code to potential users though"

Well by that argument... perhaps you should be using freebsd, slackware, solaris, or even windows? portage itself is several years more mature than paludis.

or perhaps more mature doesn't mean better.

Anonymous said...

there's a difference between being mature (as paludis) and broken by design and age (as portage)

Anonymous said...

"Well by that argument... perhaps you should be using freebsd, slackware, solaris, or even windows? portage itself is several years more mature than paludis.

or perhaps more mature doesn't mean better."

I wasn't trying to equate maturity and good as such but rather pointing out that lacking maturity can hardly be seen as a good thing. There might be other things making up for the lack of maturity but I don't see you describe those in your blog post. (I'm talking about technical aspects of the package manager here and not the much more subjective feel of the community surrounding a particular package manager).

I'd be interested to hear your take on how pkgcore is technically superior to paludis for end-users.

As your blog is about a (potential?) fork of gentoo I expect you to have thought about such a central issue quite a bit and most likely have strong arguments pointing in the direction you've decided upon.

xenoterracide said...

I'm not going to argue with technical. Because by technical you (probably) mean things developer's care about. Not necessarily we administrators (or users). EDIT: I missed the comment 'users' and don't feel like rewriting this for my misread.

Have you ever wondered why every MTA supports a sendmail like syntax? there might be a standard (couldn't say) but mostly it's because we (admins) want to be able to walk up to any machine and work on it, with out rtfm. It's the same reason we use vi(m) instead of emacs (well one of the reasons). It's the same reason windows has had trouble penetrating the server market.

We want the same UI. technical is secondary to time spent on learning another tool, unless the improvements are so much greater that the benefit largely out ways the cost in time.

there is nothing about paludis that couldn't have been done with a more emerge like syntax, which would have lowered penetration. Also the setup requirements have been atrocious. I expect to be able to install something and have a working product (that may need being tweaked). Now paludis yells at me because I haven't tweaked it.

Blatantly put this ui/config mess isn't worth my time vs any value I might get from using paludis. pkgcore works out of the box. and when it doesn't I can still use portage until it does.

I of course wouldn't recommend switching to pkgcore fully right now. It still has too many outstanding issues. But it's vision is more appropriate, and important, than being technically right (if you start with the wrong idea, you will never have the correct solution), it's existing technical issues will be fixed as it gets more mature (in time).

I believe I did mention the UI in my argument (would have to check) but I didn't go into exasperating detail.

Anonymous said...

"But it's vision is more appropriate, and important, than being technically right"

So, "I'm sorry we utterly screwed your system, but hey - at least we have a good vision"?


"if you start with the wrong idea, you will never have the correct solution"

Didn't you say you _liked_ pkgcore?

"it's existing technical issues will be fixed as it gets more mature (in time)."

See you in a few years time then.

xenoterracide said...

""if you start with the wrong idea, you will never have the correct solution"

Didn't you say you _liked_ pkgcore?"

It's a paraphrase of linus torvalds on svn vs git. Of course in his opinion being right is more important. However, he was pointing out that the idea pushing svn is wrong therefore it can never be right. I was referring to the wrong vision that paludis has, imho.

"So, "I'm sorry we utterly screwed your system, but hey - at least we have a good vision"?"

trolling? pkgcore hasn't screwed my system, it works fine. Of course running an software that's under heavy development can break a system. That comes from not running stable.

"See you in a few years time then."

See you then, we'll be here.

Ciaran said...

The big thing that should put you off pkgcore is the quality of the product... Considering the mess they made of, say, EAPI 1 (that even the most basic of testing would have caught, I might add), and considering the mess they still make of it (which users would have caught by now, were there any users), I'd be extremely wary of relying upon the code.

Anonymous said...

"I was referring to the wrong vision that paludis has, imho."

To paraphrase myself: What is that vision you see in pkgcore, and what do you find wrong with whatever vision you misguidedly seems to think paludis has (should you by chance happen to see the same vision for paludis as the actual developers has, so much the better)


"trolling?"
No thanks, I've seen enough of that in your blogpost.

"pkgcore hasn't screwed my system, it works fine."
I wasn't implying that, but based on your previous statement on the importance of a vision versus the importance on doing things correctly, I inferred that you wouldn't mind software screwing up your system as long as the developers have a vision and a plan to fix things... "in time".
If that statement holds - good luck attracting people to your upcoming fork.

"See you then, we'll be here."
For your own sanity, I hope not.

Anonymous said...

"I'm not going to argue with technical. Because by technical you (probably) mean things developer's care about. Not necessarily we administrators (or users). EDIT: I missed the comment 'users' and don't feel like rewriting this for my misread."

I was interested in knowing what makes pkgcore technically superior to paludis (or portage for that matter) seen from the user side. I'm not a gentoo user myself but I'm quite interested in the reasons for forking projects (your blog title mentions fork which is part of what attracts my curiosity).

"Have you ever wondered why every MTA supports a sendmail like syntax? there might be a standard (couldn't say) but mostly it's because we (admins) want to be able to walk up to any machine and work on it, with out rtfm. It's the same reason we use vi(m) instead of emacs (well one of the reasons). It's the same reason windows has had trouble penetrating the server market."

Supporting a sendmail syntax is needed because there's so many applications and scripts using it. So sendmail has to a large degree become an API for sending mail on unix systems. Please note that other smtp daemons offers very different configuration formats and are often divided into different processes in ways very different from sendmail. Postfix and exim (to take two fairly wellknown examples) both handle mail queues and so on in completely different ways from sendmail. Despite all the differences (even in something as important as the config files) most admins are quite happy to change from one MTA to another. As for package managers debian have been through several completely different generations and there's never been any big problems changing from one to the next. So I don't buy your argument about needing to be as similar as possible to portage.

"We want the same UI. technical is secondary to time spent on learning another tool, unless the improvements are so much greater that the benefit largely out ways the cost in time."

The time spend learning a few new options for a package manager should be fairly minimal imo. A package manager is fairly simple (from the users point of view) compared to a web server or MTA that requires fairly detailed knowledge about inner workings to configure properly. Bad documentation would be a legitimate complaint imo but it would have to be fairly horrible for most users to spend a serious amount of time figuring out how to install, upgrade and uninstall packages. I'm aware that most package managers offers many more features but most people only needs the most common features imo.

I'm struggling to see how paludis can be so hard to learn as to completely scare away users just because the options are different. Personally I see it as much more important that the package manager is always doing the right thing as it can be a real chore trying to fix problems caused by package managers. From your initial post it sounds like paludis is doing (much?) better than pkgcore in this regard.

xenoterracide said...

"If that statement holds - good luck attracting people to your upcoming fork."

well if one expects a perfect 'under development' product I guess people wouldn't use it. This is much the problem with kde4. People didn't realize is was just an api stabilization. So it's not finished thus I expect bugs. I don't run unstable in a production environment, nor would I recommend it. If I mark a release alpha I expect it to be broken, and under heavy development.

If it's beta I expect bugs but features should be there, at least the major ones.

an rc should be stable, and mostly bug free (we're just checking for anything we may have missed in the beta(s).

then we have stable which by then should be 99% bug free except for a few weird cases.

as far as pkgcore vision vs paludis vision. this blog is all about what's wrong with gentoo and the current pms. read all of it. It's all intended to be constructive criticism, and my opinions. I may not have all the facts, so I may not be right. Some of the question of right is merely opinion see vi vs emacs.

My next gen pm vision is simple. A properly coded backend for portage with the same ui, and configuration. but faster.... probably some new features too. but all what I want should be on this blog somewhere. If you still have questions I can try to answer them. (honestly none of the new pms meet these goals, pkgcore is just the closest of the 2 I looked and it should be pretty easy to fix it so that it does meet my goals.)

xenoterracide said...

"I'm struggling to see how paludis can be so hard to learn as to completely scare away users just because the options are different."

Just my opinion that ui should be similar as possible, as previous ui wasn't what was broken.


"Supporting a sendmail syntax is needed because there's so many applications and scripts using it."

sendmail is an example of course. Yes I know that conf's are different between server daemons. Currently I consider pms a convenience as opposed to mission critical, where a server daemon is not a convenience. Therefore entry to the program should be easy. (I have said it before paludis was easier this time than the last time I tried a year or so ago).

oh and the eapi 1 problem is fixed, according to a pkgcore dev. I've experienced no major problems, although I have 3 outstanding bugs that keep me from switching completely (missing features).

Anonymous said...

"sendmail is an example of course. Yes I know that conf's are different between server daemons. Currently I consider pms a convenience as opposed to mission critical, where a server daemon is not a convenience. Therefore entry to the program should be easy. (I have said it before paludis was easier this time than the last time I tried a year or so ago)."

Maybe slackware would be a good choice then. No package manager to confuse people :)

In my opinion a package manager (from the users point of view) is just a tool to help manage your system but a fairly important tool none the less. I don't think you meant to call PMs just a convenience as you might as well use autotools directly instead of a package manager in that case.

"oh and the eapi 1 problem is fixed, according to a pkgcore dev. I've experienced no major problems, although I have 3 outstanding bugs that keep me from switching completely (missing features)."

I don't think I brought up any eapi 1 problem but good to hear that it's fixed. I was somehow under the impression that there was more than one eapi 1 related bug though but every step forward benefits users.

xenoterracide said...

"I don't think I brought up any eapi 1 problem but good to hear that it's fixed. I was somehow under the impression that there was more than one eapi 1 related bug though but every step forward benefits users."

I was referring to Ciaran's comment. There may have been. I was just told they were resolved.

what I mean about a pms being a convenience is... if your pms breaks your box should still run. And you could do source installs. If a server daemon, (or any package you've installed really) breaks then you have problems. pms aren't mission critical. But I wouldn't want to have to do everything without them LFS is just not my style.

Anonymous said...

"if your pms breaks your box should still run. And you could do source installs."

Which would remove every last shred of possibility to get a package manager back on track on your system. If your package manager breaks and you do source installs around it, things will go haywire if you get that package manager up and running again - expect random breakages a long time into the future... not really what you'd expect from a "mission critical" server, no?