draft ballot: please rule on how to implement debian/rules build-arch

[Steve Langasek]

Hi folks,

Discussion appears to have died out on this thread, so it doesn't appear
anyone from the TC is waiting for more information in order to make up their
mind and I think it's (past) time to call a vote on this.

I am expanding on the original set of ballot options I'd proposed to include
additional options brought up during the discussion that I've seen support
for from the TC members. If I've missed any options that you would like to
vote for, please say so.

I've also made it explicit that each of these options are intended to be
transitional, and that we eventually want build-arch to be used
unconditionally to avoid the extra heuristics in the build tools.

BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present. This does not require either
heuristic detection (the presence or absence of arch-indep/arch-dep packages
is *definitional* of whether a separate build-arch rule is relevant, and
dpkg-buildpackage already parses debian/control), or the use of redundant
declarations (i.e, Build-Options). Should we incorporate this into the
ballot options? (For the avoidance of combinatoric proliferation, I'd
prefer to decide this question by acclamation and either incorporate it into
all the ballot options, or not.)


1) Implement support for calling 'debian/rules build-arch' in place of
'debian/rules build' by checking for the presence of the target using
'make -qn'.[1] After a transition period this target will be made
mandatory.

2) Implement support for calling 'debian/rules build-arch' with a fallback
to 'debian/rules build' by checking whether the output of the build-arch
target matches that of a dummy target.[2] After a transition period
this target will be made mandatory.

3) Implement support for calling 'debian/rules build-arch' in place of
'debian/rules build' if a Build-Options field is set in debian/control
of the source package specifying that this target is supported.[3]

4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
all packages in unstable and experimental immediately, with no fallback
if the target does not exist; requires a corresponding update to Policy
and mass updates to fix packages that fail to build as a result.

5) Implement support for calling 'debian/rules build-arch' in place of
'debian/rules build' if either it's found to be present with 'make -qn'
or if a Build-Options field is set in debian/control specifying that the
target is supported. It is recommended to only use Build-Options in
cases where autodetection fails.

6) Further Discussion


Bearing in mind the statistics from [4] indicating that option 4) will break
45% of all packages with arch-dep packages, I imagine we probably don't want
to go that route anymore. Now, maybe excluding sources that *only* build
arch-dep packages with no accompanying arch-indep packages would reduce this
percentage, but I doubt it would help enough to change anyone's vote so am
not inclined to spend more time gathering that information.

Thanks,
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org

[1] http://lists.debian.org/debian-devel/2007/07/msg00048.html
[2] http://bugs.debian.org/598534
[3] http://bugs.debian.org/229357
[4] http://lists.debian.org/debian-ctte/2011/06/msg00034.html

[Bdale Garbee]

On Sat, 23 Jul 2011 15:45:08 +0200, Steve Langasek <***@debian.org> wrote:
Non-text part: multipart/signed

Post by Steve Langasek
Discussion appears to have died out on this thread, so it doesn't appear
anyone from the TC is waiting for more information in order to make up their
mind and I think it's (past) time to call a vote on this.

I remain opposed to a manually-set Build-Options field, since I believe
that in any package someone is willing to touch, just implementing
proper separation of the targets is a far better solution.

I've been fixing the targets in all of my packages as I stumble over
them ever since the lintian check was added, and so far it has been easy
to do the right things.

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present. This does not require either
heuristic detection (the presence or absence of arch-indep/arch-dep packages
is *definitional* of whether a separate build-arch rule is relevant, and
dpkg-buildpackage already parses debian/control), or the use of redundant
declarations (i.e, Build-Options). Should we incorporate this into the
ballot options? (For the avoidance of combinatoric proliferation, I'd
prefer to decide this question by acclamation and either incorporate it into
all the ballot options, or not.)

I'd be willing to consider this, and would probably vote it ahead of the
make -qn options, though as in all the other cases, I'd be happier if
someone actually had a patch that implements this ready to go.

Post by Steve Langasek
Bearing in mind the statistics from [4] indicating that option 4) will break
45% of all packages with arch-dep packages, I imagine we probably don't want
to go that route anymore.

Actually, the more of my own packages I touch and fix, the more inclined
I am to place option [4] first on my list! Fixing source packages just
isn't that painful in practice, and I like the idea of just getting over
the policy hurdle quickly without stopgap measures.

However, of your current list of choices, I also consider the 'make -qn'
options acceptable, and would vote all options that contain the
Build-Options field alternative below further discussion.

Bdale

[Raphael Hertzog]

Post by Bdale Garbee

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present. This does not require either
heuristic detection (the presence or absence of arch-indep/arch-dep packages
is *definitional* of whether a separate build-arch rule is relevant, and
dpkg-buildpackage already parses debian/control), or the use of redundant
declarations (i.e, Build-Options). Should we incorporate this into the
ballot options? (For the avoidance of combinatoric proliferation, I'd
prefer to decide this question by acclamation and either incorporate it into
all the ballot options, or not.)

I'd be willing to consider this, and would probably vote it ahead of the
make -qn options, though as in all the other cases, I'd be happier if
someone actually had a patch that implements this ready to go.

FWIW, I would implement this if it was voted. But even if you reduce the
set of concerned packages to 2206 thanks to this, the number of instantly
broken packages would be too high to be acceptable (>600).

So we should keep auto-detection to fallback to "build" for the transition
period. Thus my favorite option is still 1 or 5 (either inconditionnaly or
only for packages witch arch-dep and arch-indep packages).

Post by Bdale Garbee
Actually, the more of my own packages I touch and fix, the more inclined
I am to place option [4] first on my list! Fixing source packages just
isn't that painful in practice, and I like the idea of just getting over
the policy hurdle quickly without stopgap measures.

One should also consider that I plan to backport dpkg (because a
substantial number of recent features will be used by packages and without
a dpkg backport it will difficult to backport all the packages which use
those features).

A squeeze system with a backported dpkg without auto-detection will fail
to build many squeeze packages even if we manage to fix them all in sid.
That would be a sad state that would require me to drop that feature in my
backported dpkg.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)

[Ian Jackson]

Post by Raphael Hertzog
A squeeze system with a backported dpkg without auto-detection will fail
to build many squeeze packages even if we manage to fix them all in sid.
That would be a sad state that would require me to drop that feature in my
backported dpkg.

The purpose of backports is to improve, not reduce, compatibility. I
think that this feature, however it is implemented, should not be
included in any backport.

Ian.

[Russ Allbery]

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present.

I think this sounds like an excellent idea.

[Ian Jackson]

Post by Russ Allbery

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present.

I think this sounds like an excellent idea.

This would be a nice way out. Do we know what proportion of packages
would fail if we did this ?

Another possibility, inspired by some bootstrapping machinery Wookey
told me about, would be this:
DEB_BUILD_OPTIONS=build-arch-only debian/rules build
Either as a transitional arrangement, or permanently. I mention it
here because it seems like a plausible idea and we should consider
whether we like it.

Personally I really dislike all the make-based heuristic autodetection
and will vote any of those below FD (although I expect to be
outvoted).

Maybe we're going to end up with "break n% of the archive" as the
least-hated answer...

Ian.

[Steve Langasek]

Post by Ian Jackson

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present.

This would be a nice way out. Do we know what proportion of packages
would fail if we did this ?
Another possibility, inspired by some bootstrapping machinery Wookey
DEB_BUILD_OPTIONS=build-arch-only debian/rules build
Either as a transitional arrangement, or permanently. I mention it
here because it seems like a plausible idea and we should consider
whether we like it.
Personally I really dislike all the make-based heuristic autodetection
and will vote any of those below FD (although I expect to be
outvoted).
Maybe we're going to end up with "break n% of the archive" as the
least-hated answer...

Well, I was planning on voting "break n% of the archive" before FD for the
current value of n. :)

[Steve Langasek]

Post by Steve Langasek

Post by Ian Jackson
Maybe we're going to end up with "break n% of the archive" as the
least-hated answer...

Well, I was planning on voting "break n% of the archive" before FD for the

^^^^^^ below

Post by Steve Langasek
current value of n. :)

... and I would want to consult the release team before voting other values
of n above FD.

[Guillem Jover]

Hi!

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present. This does not require either
heuristic detection (the presence or absence of arch-indep/arch-dep packages
is *definitional* of whether a separate build-arch rule is relevant, and
dpkg-buildpackage already parses debian/control), or the use of redundant
declarations (i.e, Build-Options). Should we incorporate this into the
ballot options? (For the avoidance of combinatoric proliferation, I'd
prefer to decide this question by acclamation and either incorporate it into
all the ballot options, or not.)

In the thread in debian-policy I also proposed [0] another small variation
for this, which would imply a two stage transition, reducing the immediate
simultaneous FTBFS. Jakub Wilk provided numbers [1] for the possible FTBFS
on the non-staged scenario.

thanks,
guillem

[0] <http://lists.debian.org/debian-policy/2011/06/msg00026.html>
[1] <http://lists.debian.org/debian-policy/2011/06/msg00018.html>

[Ian Jackson]

Post by Guillem Jover
In the thread in debian-policy I also proposed [0] another small variation
for this, which would imply a two stage transition, reducing the immediate
simultaneous FTBFS. Jakub Wilk provided numbers [1] for the possible FTBFS
on the non-staged scenario.

...

Post by Guillem Jover
[0] <http://lists.debian.org/debian-policy/2011/06/msg00026.html>
[1] <http://lists.debian.org/debian-policy/2011/06/msg00018.html>
- Lintian doesn't warn about missing build-* targets yet, so many
maintainers are not aware that their package are affected by this issue.

Is this still true ? If so I think we should definitely fix this
right away. No matter what the eventual transition plan is, we
definitely want this to be at least a Lintian warning. So a change to
Lintian now to warn about this situation is a no-brainer I think.

Ian.

[Russ Allbery]

Post by Ian Jackson

Post by Guillem Jover
- Lintian doesn't warn about missing build-* targets yet, so many
maintainers are not aware that their package are affected by this issue.

Is this still true ?

No, Lintian has been warning since 2.5.1.

[Bdale Garbee]

Post by Ian Jackson

Post by Guillem Jover
- Lintian doesn't warn about missing build-* targets yet, so many
maintainers are not aware that their package are affected by this issue.

Is this still true ?

No. Lintian has been warning me about the missing targets .. I'm not
sure when that changed, but I believe it's doing the right things now.

Bdale

[Christian PERRIER]

Post by Bdale Garbee

Post by Ian Jackson

Post by Guillem Jover
- Lintian doesn't warn about missing build-* targets yet, so many
maintainers are not aware that their package are affected by this issue.

Is this still true ?

No. Lintian has been warning me about the missing targets .. I'm not
sure when that changed, but I believe it's doing the right things now.

And, as a Stupid And Clueless Maintainer, I have to say that, up to
now, I just blindly adopted what lintian suggested to me....assuming
that Those Who Know know what they're doing and that adding these
target is What Should Be Done...:-)

Or, even more simpler, I try to use dh7-style minimal rules files and
let debhelper handle all the magic..:-)

Still, nearly each package of mine I worked on recently had the
"issue" of not having build-arch and build-any, so I suspect this is
the case for many pkgs in the archive.

[Steve Langasek]

Post by Guillem Jover

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present. This does not require either
heuristic detection (the presence or absence of arch-indep/arch-dep packages
is *definitional* of whether a separate build-arch rule is relevant, and
dpkg-buildpackage already parses debian/control), or the use of redundant
declarations (i.e, Build-Options). Should we incorporate this into the
ballot options? (For the avoidance of combinatoric proliferation, I'd
prefer to decide this question by acclamation and either incorporate it into
all the ballot options, or not.)

In the thread in debian-policy I also proposed [0] another small variation
for this, which would imply a two stage transition, reducing the immediate
simultaneous FTBFS. Jakub Wilk provided numbers [1] for the possible FTBFS
on the non-staged scenario.
[0] <http://lists.debian.org/debian-policy/2011/06/msg00026.html>
[1] <http://lists.debian.org/debian-policy/2011/06/msg00018.html>

I think this is too much hair splitting. To be clear, I was proposing a
change to our *desired end state*, but that the ballot options to choose
between would otherwise remain the same - and I'm still in favor of getting
the ball rolling using the make -qn heuristic check. And if we're using the
heuristics to reduce the initial FTBFS set, another intermediate stage is
unnecessary.

[Carsten Hey]

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present. This does not require either
heuristic detection (the presence or absence of arch-indep/arch-dep packages
is *definitional* of whether a separate build-arch rule is relevant, and
dpkg-buildpackage already parses debian/control), or the use of redundant
declarations (i.e, Build-Options).

An other option is to release a new Debian Policy version (maybe a major
one), including either the requirement for source packages with
arch-indep and arch-dep packages to provide build-arch (what you
described above), or require all packages to provide build-arch as
target; and during a transition period, let dpkg-buildpackage use the
Standards-Version of a package to decide whether the build-arch target
should be used.

Carsten

[Steve Langasek]

Post by Carsten Hey

Post by Steve Langasek
BTW, another option for the long-term solution which we haven't really
addressed head-on is that dpkg-buildpackage could detect whether both
arch-indep and arch-dep packages are present in debian/control, and use
build-arch *only* when both are present. This does not require either
heuristic detection (the presence or absence of arch-indep/arch-dep packages
is *definitional* of whether a separate build-arch rule is relevant, and
dpkg-buildpackage already parses debian/control), or the use of redundant
declarations (i.e, Build-Options).

An other option is to release a new Debian Policy version (maybe a major
one), including either the requirement for source packages with
arch-indep and arch-dep packages to provide build-arch (what you
described above), or require all packages to provide build-arch as
target; and during a transition period, let dpkg-buildpackage use the
Standards-Version of a package to decide whether the build-arch target
should be used.

I would not vote such an option above FD and would expect it to be entirely
dominated by the ballot option to use a Build-Options flag instead; so I
won't include this on a proposed ballot unless another member of the TC says
this is interesting to them.

I think there is a very clear consensus in the community that the value of
the Standards-Version field should never affect the behavior of build tools.

[Raphael Hertzog]

Hi,

Post by Steve Langasek
Discussion appears to have died out on this thread, so it doesn't appear
anyone from the TC is waiting for more information in order to make up their
mind and I think it's (past) time to call a vote on this.

Can we get a vote and a decision, please?

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/

[Roger Leigh]

Post by Raphael Hertzog
Hi,

Post by Steve Langasek
Discussion appears to have died out on this thread, so it doesn't appear
anyone from the TC is waiting for more information in order to make up their
mind and I think it's (past) time to call a vote on this.

Can we get a vote and a decision, please?

At this point, we have one working and well tested solution. Is there
any point in waiting on the TC at this point given that it's really the
only sensible choice (as in, it's been tested on the whole archive and
we know its impact is very low, and we don't have any alternative
patches at this point which offer a better solution). And given that
we really want this in place well before wheezy, to permit sufficient
time for testing, further time lost waiting is not desirable.


Regards,
Roger

[Raphael Hertzog]

Hi,

Post by Roger Leigh
At this point, we have one working and well tested solution.

Which one are you referring to ?

Post by Roger Leigh
Is there any point in waiting on the TC at this point given that it's
really the only sensible choice (as in, it's been tested on the whole
archive and we know its impact is very low, and we don't have any
alternative patches at this point which offer a better solution).

IIRC, the solution that you're referring to is one that Guillem was not
really pleased with. So that makes a reason for a TC ruling.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/

[Roger Leigh]

Post by Raphael Hertzog
Hi,

Post by Roger Leigh
At this point, we have one working and well tested solution.

Which one are you referring to ?

I'm referring to using "make -qn" (with or without the additional
presence of the Build-Options field). Either would do the job, and
Build-Options would cater for packages where the autodetection breaks
the build. Given the tiny number of packages affected, this isn't a
big deal IMO.

Post by Raphael Hertzog

Post by Roger Leigh
Is there any point in waiting on the TC at this point given that it's
really the only sensible choice (as in, it's been tested on the whole
archive and we know its impact is very low, and we don't have any
alternative patches at this point which offer a better solution).

IIRC, the solution that you're referring to is one that Guillem was not
really pleased with. So that makes a reason for a TC ruling.

OK. Guillem (CC'd), are there any existing viable alternatives to
"make -qn" (with or without Build-Options) which you are happy with?
Note that the purpose of this change is not a "perfect" long-term
solution, but a "good enough" solution which will permit the adoption
of build-arch and build-indep without a flag day which would break
everything. It can be removed once wheezy is released, following
which the targets can be made mandatory. Of course, if a better
method of target autodetection is found in the interim, we can
adopt that instead. This would strictly be to enable the
transition.

As I see it, we've been waiting for such a perfect solution for over
eight years, and because of that, it's never happened. A less-than-
perfect solution would have allowed this to be done years ago. We
need it working for wheezy, and in the absence of a better solution
having your consent to go with the "make -qn" solution would at
least allow some progress to be made. It would only need to be
there until the release of wheezy.


Regards,
Roger

[Guillem Jover]

Hi!

Post by Roger Leigh

Post by Raphael Hertzog

Post by Roger Leigh
At this point, we have one working and well tested solution.

Which one are you referring to ?

I'm referring to using "make -qn" (with or without the additional
presence of the Build-Options field). Either would do the job, and
Build-Options would cater for packages where the autodetection breaks
the build. Given the tiny number of packages affected, this isn't a
big deal IMO.

I think Raphaël thought you were talking exclusively about a
Build-Options based solution.

Post by Roger Leigh

Post by Raphael Hertzog

Post by Roger Leigh
Is there any point in waiting on the TC at this point given that it's
really the only sensible choice (as in, it's been tested on the whole
archive and we know its impact is very low, and we don't have any
alternative patches at this point which offer a better solution).

IIRC, the solution that you're referring to is one that Guillem was not
really pleased with. So that makes a reason for a TC ruling.

OK. Guillem (CC'd), are there any existing viable alternatives to
"make -qn" (with or without Build-Options) which you are happy with?
Note that the purpose of this change is not a "perfect" long-term
solution, but a "good enough" solution which will permit the adoption
of build-arch and build-indep without a flag day which would break
everything. It can be removed once wheezy is released, following
which the targets can be made mandatory. Of course, if a better
method of target autodetection is found in the interim, we can
adopt that instead. This would strictly be to enable the
transition.

I think I've stated my take on this at least in #604397. In any case
I'm fine with mostly any solution ranging (in no particular order)
from a global activation of the targets on a flag day to incremental
activation depending on the Build-Depends(-Indep) fields, to
autodetection heuristics, or combinations of those, as long as any
such temporary hack is confined to dpkg itself. But certainly I'm not
fine with a Build-Options/Build-Features kind of field solution that
percolates into packages, when they should just be fixed to support
build-arch/build-indep.

Post by Roger Leigh
As I see it, we've been waiting for such a perfect solution for over
eight years, and because of that, it's never happened. A less-than-
perfect solution would have allowed this to be done years ago. We
need it working for wheezy, and in the absence of a better solution
having your consent to go with the "make -qn" solution would at
least allow some progress to be made. It would only need to be
there until the release of wheezy.

IMO the main problem here has been not using these eight years to just
start deploying build-arch/build-indep into packages independent on how
or when dpkg was going to start using them, a lintian warning back then
would have made wonders and by now the archive would be mostly covered.

In any case I don't think I've ever opposed «make -qn», all the
contrary!

thanks,
guillem

[Raphael Hertzog]

Hi,

Post by Guillem Jover
I think I've stated my take on this at least in #604397. In any case
I'm fine with mostly any solution ranging (in no particular order)
from a global activation of the targets on a flag day to incremental
activation depending on the Build-Depends(-Indep) fields, to
autodetection heuristics, or combinations of those, as long as any
such temporary hack is confined to dpkg itself. But certainly I'm not
fine with a Build-Options/Build-Features kind of field solution that
percolates into packages, when they should just be fixed to support
build-arch/build-indep.

[...]

Post by Guillem Jover
In any case I don't think I've ever opposed «make -qn», all the
contrary!

Ok, then I suggest to go ahead with the attached patch. If I don't hear
any objections, I'll push it later this week.

This one uses build-arch/build-indep by default but falls back to build
if "make -qn" says the target is not supported.

The number of packages broken by this change is relatively small according
to the rebuilds of Roger (~100 out of all the arch-any packages, probably
twice that big if we include arch-all packages). And this number might
have decreased since the tests given that lintian has been advising
maintainers to add the build-arch/build-indep targets.

This means that build-arch/build-indep ought to be mandatory for all
packages from now on.

(We could have argued to use build-arch/build-indep only for packages
which build both arch any&all binaries but that would create an assymetry
with the way the binary target is handled and IMO it's best avoided)

Post by Guillem Jover

Post by Roger Leigh
least allow some progress to be made. It would only need to be
there until the release of wheezy.

I think you're rather optimistic, this transition will not be completed
before the release of wheezy.

Cheers,

[Russ Allbery]

Post by Raphael Hertzog
Ok, then I suggest to go ahead with the attached patch. If I don't hear
any objections, I'll push it later this week.
This one uses build-arch/build-indep by default but falls back to build
if "make -qn" says the target is not supported.
The number of packages broken by this change is relatively small
according to the rebuilds of Roger (~100 out of all the arch-any
packages, probably twice that big if we include arch-all packages). And
this number might have decreased since the tests given that lintian has
been advising maintainers to add the build-arch/build-indep targets.
This means that build-arch/build-indep ought to be mandatory for all
packages from now on.

This sounds like the right move to me. I think we (the tech-ctte) should
have a vote and make it official. (In fact, I think we're probably long
overdue for voting on this.)

[Don Armstrong]

Post by Russ Allbery

Post by Raphael Hertzog
Ok, then I suggest to go ahead with the attached patch. If I don't hear
any objections, I'll push it later this week.
This one uses build-arch/build-indep by default but falls back to build
if "make -qn" says the target is not supported.
The number of packages broken by this change is relatively small
according to the rebuilds of Roger (~100 out of all the arch-any
packages, probably twice that big if we include arch-all packages). And
this number might have decreased since the tests given that lintian has
been advising maintainers to add the build-arch/build-indep targets.
This means that build-arch/build-indep ought to be mandatory for all
packages from now on.

This sounds like the right move to me. I think we (the tech-ctte) should
have a vote and make it official. (In fact, I think we're probably long
overdue for voting on this.)

I believe we're just missing someone to write up a ballot with the
options and call for a vote.


Don Armstrong

[Guillem Jover]

Post by Raphael Hertzog
Ok, then I suggest to go ahead with the attached patch. If I don't hear
any objections, I'll push it later this week.
commit 557b5809eef183da2e4907e994dabaa9273e7e0b
Date: Tue Jan 24 11:59:44 2012 +0100
dpkg-buildpackage: use build-arch and build-indep targets of debian/rules
'build-arch' is used when building only arch-any binaries (-B)
while 'build-indep' is used when building only arch-all binaries (-A).
To avoid breaking too many packages, dpkg-buildpackages verifies that
those targets are implemented by calling “make -f debian/rules -qn
<target>” and ensuring that it doesn't fail with exit code 2. Otherwise
it fallsback to using the 'build' target.
This fallback is a temporary measure until all packages have been
converted to properly support the build-arch and build-indep targets.

Actually thinking about this, making this temporary will imply that
once this would get removed old/external packages would stop building
which is not acceptable. So I think we might need to carry this for
quite some time (if not indefinitely).

Post by Raphael Hertzog
diff --git a/man/dpkg-buildpackage.1 b/man/dpkg-buildpackage.1
index 5410a20..f7d1cf1 100644
--- a/man/dpkg-buildpackage.1
+++ b/man/dpkg-buildpackage.1
@@ -27,12 +27,13 @@ It calls \fBdpkg\-source \-b\fP to generate the source package (unless
a binary\-only build has been requested with \fB\-b\fP, \fB\-B\fP or
\fB\-A\fP).
.IP \fB5.\fP 3
-It calls \fBdebian/rules\fP \fBbuild\fP followed by
-\fBfakeroot debian/rules\fP \fIbinary-target\fP (unless a source-only
-build has been requested with \fB\-S\fP). Note that \fIbinary-target\fR is
-either \fBbinary\fP (default case, or if \fB\-b\fP is specified)
-or \fBbinary\-arch\fP (if \fB\-B\fP is specified) or \fBbinary\-indep\fP
-(if \fB\-A\fP is specified).
+It calls \fBdebian/rules\fP \fIbuild\-target\fP followed by
+\fBfakeroot debian/rules\fP \fIbinary\-target\fP (unless a source-only
+build has been requested with \fB\-S\fP). Note that \fIbuild\-target\fR
+and \fIbinary\-target\fP are either \fBbuild\fP and \fBbinary\fP (default
+case, or if \fB\-b\fP is specified), or \fBbuild\-arch\fP and
+\fBbinary\-arch\fP (if \fB\-B\fP is specified), or \fBbuild\-indep\fP and
+\fBbinary\-indep\fP (if \fB\-A\fP is specified).

foo-target do not need the ‘-’ escaped as they are variable text not
to be used as input on the command line for example.

Post by Raphael Hertzog
.IP \fB6.\fP 3
It calls \fBgpg\fP to sign the \fB.dsc\fP file (if any, unless
\fB\-us\fP is specified).
@@ -241,6 +242,13 @@ exported compiler flags (\fBCFLAGS\fP, \fBCXXFLAGS\fP, \fBFFLAGS\fP,
\fBCPPFLAGS\fP and \fBLDFLAGS\fP) with values as returned
by \fBdpkg\-buildflags\fP. This is no longer the case.
.
+.SH BACKWARD COMPATIBILITY
+\fBdpkg\-buildpackage\fP is using the \fBbuild-arch\fP and
+\fBbuild-indep\fP targets since version 1.16.2. Those targets are thus
+mandatory. But to avoid breakages of existing packages, and ease
+the transition, it will fallback to using the \fBbuild\fP target
+if \fBmake -f debian/rules -qn\fP \fIbuild-target\fP returns 2 as
+exit code.

build-arch/build-indep/-f/-qn need ‘-’ escaped. Also given my comment
above about the backwards compability issues, I'd remove the transition
part.

Post by Raphael Hertzog
.SH BUGS
It should be possible to specify spaces and shell metacharacters in
and initial arguments for
diff --git a/scripts/dpkg-buildpackage.pl b/scripts/dpkg-buildpackage.pl
index 50e6170..e1dd538 100755
--- a/scripts/dpkg-buildpackage.pl
+++ b/scripts/dpkg-buildpackage.pl
@@ -385,6 +390,7 @@ if ($call_target) {
unless ($noclean) {
}
+
unless (build_binaryonly) {
warning(_g("it is a bad idea to generate a source package " .
"without cleaning up first, it might contain undesired " .

Why the spurious newlines?

Post by Raphael Hertzog
@@ -393,10 +399,30 @@ unless (build_binaryonly) {
chdir($dir) or syserr("chdir $dir");
}
+
+ # Verify that build-{arch,indep} are supported. If not, fallback to build.
+ # This is a temporary measure to avoid breaking too many packages on a flag day.

These comment lines could be wrapped to < 80 chars. And again I'd
remove the comment about this being a temporary measure.

Post by Raphael Hertzog
unless (build_sourceonly) {
}
+

Spurious new line.


In any case, looks good, thanks! So, besides those comments:

Acked-by: Guillem Jover <***@debian.org>

thanks,
guillem

[Raphael Hertzog]

Post by Guillem Jover

Post by Raphael Hertzog
This fallback is a temporary measure until all packages have been
converted to properly support the build-arch and build-indep targets.

Actually thinking about this, making this temporary will imply that
once this would get removed old/external packages would stop building
which is not acceptable. So I think we might need to carry this for
quite some time (if not indefinitely).

IIRC you said before that a solution involving a flag day was ok for you.
And now you care about compatibility of old/external packages? :-)

In any case, I really care about this being temporary because, as we
saw, the make -qn call can have undesired side effects and I don't want
to keep this source of problem indefinitely.

I have no problem to keep it for very long, but I would like to document
it as temporary (or, what I considered initially, not document the
existence of this kludge at all).
Thanks for the review. I fixed everything except the documentation
about the temporary nature of this work-around.

Updated patch attached.

Cheers,

[Guillem Jover]

Post by Raphael Hertzog

Post by Guillem Jover

Post by Raphael Hertzog
This fallback is a temporary measure until all packages have been
converted to properly support the build-arch and build-indep targets.

Actually thinking about this, making this temporary will imply that
once this would get removed old/external packages would stop building
which is not acceptable. So I think we might need to carry this for
quite some time (if not indefinitely).

IIRC you said before that a solution involving a flag day was ok for you.
And now you care about compatibility of old/external packages? :-)
In any case, I really care about this being temporary because, as we
saw, the make -qn call can have undesired side effects and I don't want
to keep this source of problem indefinitely.
I have no problem to keep it for very long, but I would like to document
it as temporary (or, what I considered initially, not document the
existence of this kludge at all).

Yes, I reorganized my thoughts while sleeping, in essence I'm mostly
concerned about the *possibility* of building/using old sources and
binaries, and as long as there's a way, even if behind --force- or
other similar options I don't have any problem with that.

For this case we do have -T so the previous behaviour can always be
somehow emulated manually, it might make sense to move the heuristic to
an option that can be used explicitly once we disable it by default.
On the other hand dpkg-buildpackage is not the official interface to
build packages, debian/rules is, so the builder can always use that.

Post by Raphael Hertzog
+ # Verify that build-{arch,indep} are supported. If not, fallback to build.
+ # This is a temporary measure to not break too many packages on a flag day.

It still would be nice to wrap this under 80 chars. :)

thanks,
guillem

[Roger Leigh]

Post by Raphael Hertzog

Post by Raphael Hertzog
This fallback is a temporary measure until all packages have been
converted to properly support the build-arch and build-indep targets.

Updated patch attached.
+ warning(_g("%s must be updated to support the 'build-arch' and " .
+ "'build-indep' targets (at least '%s' seems to be " .

I was wondering if we could make the emphasis here stronger to
encourage adoption in a timely manner. Could we maybe add an
0x07 (\a) in the string to make it beep, and add a sleep 15
afterward to ensure that the warning is overly obtrusive?
(Slightly annoying, I know, which is the intention!)

I also wonder, once the patch is in place, how aggressively we
could push adoption in unstable. With sufficient motivation and
aggressive NMUs, could we get this done for wheezy and release
with the autodetection removed? i.e. migrate the majority of
packages, and then have the "flag day" and fix all remaining
packages with NMUs?


Regards,
Roger

[Raphael Hertzog]

Post by Roger Leigh

Post by Raphael Hertzog

Post by Raphael Hertzog
This fallback is a temporary measure until all packages have been
converted to properly support the build-arch and build-indep targets.

Updated patch attached.
+ warning(_g("%s must be updated to support the 'build-arch' and " .
+ "'build-indep' targets (at least '%s' seems to be " .

I was wondering if we could make the emphasis here stronger to
encourage adoption in a timely manner. Could we maybe add an
0x07 (\a) in the string to make it beep, and add a sleep 15
afterward to ensure that the warning is overly obtrusive?
(Slightly annoying, I know, which is the intention!)

No, I'm not going to add anything like that.

Post by Roger Leigh
I also wonder, once the patch is in place, how aggressively we
could push adoption in unstable. With sufficient motivation and
aggressive NMUs, could we get this done for wheezy and release
with the autodetection removed? i.e. migrate the majority of
packages, and then have the "flag day" and fix all remaining
packages with NMUs?

I don't believe that this would be realistic, but I'm certainly not going
to forbid you to aim for this. There will be no "flag day" but if you
manage to convince the release team that it's a worthwile realease goal,
then you can have relaxed NMU rules...

In any case, whatever progress you managed to do, I fully expect to
release dpkg with this compatibility code as users outside of Debian also
need time to transition and update.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/