[PATCH] Make output of mdate-sh deterministic

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH] Make output of mdate-sh deterministic

Reiner Herrmann
mdate-sh pretty-prints the modification time of a file.
But it's output can vary depending on the timezone of
the caller. Someone in timezone GMT-12 will get a different
result (day) than someone in timezone GMT+12.

As this output is also used to create/update stamp files,
which influence the further build process, the build result
can vary.
To enable reproducible builds and to have a more deterministic
build behavior, this change fixes the timezone to UTC.

Signed-off-by: Reiner Herrmann <[hidden email]>
---
 lib/mdate-sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/mdate-sh b/lib/mdate-sh
index b793600..3e7d6ae 100755
--- a/lib/mdate-sh
+++ b/lib/mdate-sh
@@ -1,7 +1,7 @@
 #!/bin/sh
 # Get modification time of a file or directory and pretty-print it.
 
-scriptversion=2010-08-21.06; # UTC
+scriptversion=2015-04-09.19; # UTC
 
 # Copyright (C) 1995-2015 Free Software Foundation, Inc.
 # written by Ulrich Drepper <[hidden email]>, June 1995
@@ -74,6 +74,10 @@ export LC_ALL
 LC_TIME=C
 export LC_TIME
 
+# Use UTC to get reproducible result
+TZ=UTC
+export TZ
+
 # GNU ls changes its time format in response to the TIME_STYLE
 # variable.  Since we cannot assume 'unset' works, revert this
 # variable to its documented default.
--
2.1.4


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Make output of mdate-sh deterministic

Mathieu Lirzin
Hello,

Reiner Herrmann <[hidden email]> writes:

> mdate-sh pretty-prints the modification time of a file.
> But it's output can vary depending on the timezone of
> the caller. Someone in timezone GMT-12 will get a different
> result (day) than someone in timezone GMT+12.
>
> As this output is also used to create/update stamp files,
> which influence the further build process, the build result
> can vary.
> To enable reproducible builds and to have a more deterministic
> build behavior, this change fixes the timezone to UTC.
>
> Signed-off-by: Reiner Herrmann <[hidden email]>
> ---
>  lib/mdate-sh | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>

Applied with slight modifications in commit
7c25c996d1c7c212a5981aa0e9c4434b6f33f7b8

Thanks.

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Make output of mdate-sh deterministic

Mattia Rizzolo
On Fri, Sep 15, 2017 at 12:34:54PM +0200, Mathieu Lirzin wrote:
> Applied with slight modifications in commit
> 7c25c996d1c7c212a5981aa0e9c4434b6f33f7b8

Great, thank you!

--
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
more about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Make output of mdate-sh deterministic

Eric Dorland-4
In reply to this post by Mathieu Lirzin
This is a good change but it's not enough unfortunately to make it
reproducible.  mdate-sh also needs to support SOURCE_DATE_EPOCH. I'm
working on a patch for that.

* Mathieu Lirzin ([hidden email]) wrote:

> Hello,
>
> Reiner Herrmann <[hidden email]> writes:
>
> > mdate-sh pretty-prints the modification time of a file.
> > But it's output can vary depending on the timezone of
> > the caller. Someone in timezone GMT-12 will get a different
> > result (day) than someone in timezone GMT+12.
> >
> > As this output is also used to create/update stamp files,
> > which influence the further build process, the build result
> > can vary.
> > To enable reproducible builds and to have a more deterministic
> > build behavior, this change fixes the timezone to UTC.
> >
> > Signed-off-by: Reiner Herrmann <[hidden email]>
> > ---
> >  lib/mdate-sh | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
>
> Applied with slight modifications in commit
> 7c25c996d1c7c212a5981aa0e9c4434b6f33f7b8
>
> Thanks.
>
--
Eric Dorland <[hidden email]>
43CF 1228 F726 FD5B 474C  E962 C256 FBD5 0022 1E93

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: bug#20314: [PATCH] Make output of mdate-sh deterministic

Mathieu Lirzin
Hello Eric,

Eric Dorland <[hidden email]> writes:

> This is a good change but it's not enough unfortunately to make it
> reproducible.  mdate-sh also needs to support SOURCE_DATE_EPOCH. I'm
> working on a patch for that.

Thanks for working on that.

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Reply | Threaded
Open this post in threaded view
|

Re: bug#20314: [PATCH] Make output of mdate-sh deterministic

Paul Eggert
Unfortunately that patch to Automake's mdate-sh is not portable, as TZ='UTC' is
not a portable setting for the TZ environment variable. POSIX says you're
supposed to use something like TZ='UTC0' instead. Although TZ='UTC' works when
glibc is used, this is not necessarily true on other POSIX platforms.

I noticed this problem when recent Automake changes were merged into Gnulib, and
installed the attached patch to the Automake master branch to fix this. Please
review any other patches you may be using for reproducible builds, and fix them
to use TZ='UTC0' instead of TZ='UTC'. Thanks.

For reference, here's the POSIX spec for TZ:

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03

and look for "TZ".

0001-lib-mdate.sh-TZ-Use-portable-setting.patch (941 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: bug#20314: [PATCH] Make output of mdate-sh deterministic

Mathieu Lirzin
Hello Paul,

Paul Eggert <[hidden email]> writes:

> Unfortunately that patch to Automake's mdate-sh is not portable, as
> TZ='UTC' is not a portable setting for the TZ environment
> variable. POSIX says you're supposed to use something like TZ='UTC0'
> instead. Although TZ='UTC' works when glibc is used, this is not
> necessarily true on other POSIX platforms.
>
> I noticed this problem when recent Automake changes were merged into
> Gnulib, and installed the attached patch to the Automake master branch
> to fix this. Please review any other patches you may be using for
> reproducible builds, and fix them to use TZ='UTC0' instead of
> TZ='UTC'. Thanks.
>
> For reference, here's the POSIX spec for TZ:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03
>
> and look for "TZ".

Thanks for fixing that and for the reference.

> From 5b240b3b36766045a47a6ad89ae5f4550e81d534 Mon Sep 17 00:00:00 2001
> From: Paul Eggert <[hidden email]>
> Date: Thu, 21 Sep 2017 20:08:48 -0700
> Subject: [PATCH] * lib/mdate.sh (TZ): Use portable setting.

nitpick: I would prefer your future commit messages to include a summary
line before the list of modified files.

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Reply | Threaded
Open this post in threaded view
|

Re: bug#20314: [PATCH] Make output of mdate-sh deterministic

Bruno Haible
In reply to this post by Paul Eggert
Paul Eggert wrote:
> Unfortunately that patch to Automake's mdate-sh is not portable, as TZ='UTC' is
> not a portable setting for the TZ environment variable. POSIX says you're
> supposed to use something like TZ='UTC0' instead. Although TZ='UTC' works when
> glibc is used, this is not necessarily true on other POSIX platforms.

Gnulib also supports MSVC, which interprets the TZ environment variable in its
own way [1][2]. From this doc and from POSIX [3], it looks to me that
  UTC0
  GMT0
  GMT+0
  GMT-0
would all be equivalent and portable. Can you confirm this? Or does one of
these TZ values happen to enable DST?

Bruno

[1] https://msdn.microsoft.com/en-us/library/aa273389.aspx
[2] https://msdn.microsoft.com/en-us/library/90s5c885.aspx
[3] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html


Reply | Threaded
Open this post in threaded view
|

Re: bug#20314: [PATCH] Make output of mdate-sh deterministic

Paul Eggert
On 09/22/2017 02:57 AM, Bruno Haible wrote:
> Gnulib also supports MSVC, which interprets the TZ environment variable in its
> own way [1][2]. From this doc and from POSIX [3], it looks to me that
>    UTC0
>    GMT0
>    GMT+0
>    GMT-0
> would all be equivalent and portable. Can you confirm this?

All of them would use UTC (the first one with the abbreviation "UTC",
the others with "GMT"). Other variants would work as well, e.g.,
"UTC-00:00:00". I usually use "UTC0" as it's simplest and most
technically correct (GMT stopped being standardized many years ago).