excluding intermediate flex test suite files from distribution?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

excluding intermediate flex test suite files from distribution?

Will Estes
All,

The flex program includes a test suite to enable testing of the version of flex built in the flex tree. From a distributed tar ball (made with "make dist"), it is possible to build flex without needing to have flex already installed because automake includes the intermediate files in the distribution.

However, the test suite should not include the intermediate .c (and c++) files because the point of the test suite is to test the flex binary built in the tree, so that binary should build the intermediate .c (and c++) files.

How do I do this? I've been playing with it and have not been able to come up with a solution. At most, I'm able to not include some generated header files, which then really confuses the test suite in the generated tar ball.

What other information / examples can I provide to make this clear?

Thanks!

--Will


--
Will Estes
[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excluding intermediate flex test suite files from distribution?

Nick Bowler
Hello,

On 2015-11-12, Will Estes <[hidden email]> wrote:

> The flex program includes a test suite to enable testing of the version of
> flex built in the flex tree. From a distributed tar ball (made with "make
> dist"), it is possible to build flex without needing to have flex already
> installed because automake includes the intermediate files in the
> distribution.
>
> However, the test suite should not include the intermediate .c (and c++)
> files because the point of the test suite is to test the flex binary built
> in the tree, so that binary should build the intermediate .c (and c++)
> files.
>
> How do I do this? I've been playing with it and have not been able to come
> up with a solution. At most, I'm able to not include some generated header
> files, which then really confuses the test suite in the generated tar ball.
>
> What other information / examples can I provide to make this clear?

If you can provide a (short!) example Makefile.am, members of this list
may be able to suggest specific changes.

The Automake manual has a chapter on how the distribution is built[1].

A possible solution (not sure if it will work for you, you may have to
experiment a bit):

  - Prevent Automake from distributing any of the testsuite-related
    flex source files (should be achievable using nodist_).
  - Manually include only the files you actually want, e.g., by using
    EXTRA_DIST.

Using a dist-hook may be helpful if EXTRA_DIST is not expressive enough.

[1] https://gnu.org/software/automake/manual/automake.html#Dist

Cheers,
  Nick

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excluding intermediate flex test suite files from distribution?

Will Estes
Yeah, I'll work up a small example because the entire test suite is large and there's a lot of stuff that follows the same pattern.

On Thursday, 12 November 2015,  1:55 pm -0500, Nick Bowler <[hidden email]> wrote:

> Hello,
>
> On 2015-11-12, Will Estes <[hidden email]> wrote:
> > The flex program includes a test suite to enable testing of the version of
> > flex built in the flex tree. From a distributed tar ball (made with "make
> > dist"), it is possible to build flex without needing to have flex already
> > installed because automake includes the intermediate files in the
> > distribution.
> >
> > However, the test suite should not include the intermediate .c (and c++)
> > files because the point of the test suite is to test the flex binary built
> > in the tree, so that binary should build the intermediate .c (and c++)
> > files.
> >
> > How do I do this? I've been playing with it and have not been able to come
> > up with a solution. At most, I'm able to not include some generated header
> > files, which then really confuses the test suite in the generated tar ball.
> >
> > What other information / examples can I provide to make this clear?
>
> If you can provide a (short!) example Makefile.am, members of this list
> may be able to suggest specific changes.
>
> The Automake manual has a chapter on how the distribution is built[1].
>
> A possible solution (not sure if it will work for you, you may have to
> experiment a bit):
>
>   - Prevent Automake from distributing any of the testsuite-related
>     flex source files (should be achievable using nodist_).
>   - Manually include only the files you actually want, e.g., by using
>     EXTRA_DIST.
>
> Using a dist-hook may be helpful if EXTRA_DIST is not expressive enough.
>
> [1] https://gnu.org/software/automake/manual/automake.html#Dist
>
> Cheers,
>   Nick
>

--
Will Estes
[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excluding intermediate flex test suite files from distribution?

Will Estes
In reply to this post by Nick Bowler
On Thursday, 12 November 2015,  1:55 pm -0500, Nick Bowler <[hidden email]> wrote:

> Hello,
>
> On 2015-11-12, Will Estes <[hidden email]> wrote:
> > The flex program includes a test suite to enable testing of the version of
> > flex built in the flex tree. From a distributed tar ball (made with "make
> > dist"), it is possible to build flex without needing to have flex already
> > installed because automake includes the intermediate files in the
> > distribution.
> >
> > However, the test suite should not include the intermediate .c (and c++)
> > files because the point of the test suite is to test the flex binary built
> > in the tree, so that binary should build the intermediate .c (and c++)
> > files.
> >
> > How do I do this? I've been playing with it and have not been able to come
> > up with a solution. At most, I'm able to not include some generated header
> > files, which then really confuses the test suite in the generated tar ball.
> >
> > What other information / examples can I provide to make this clear?
>
> If you can provide a (short!) example Makefile.am, members of this list
> may be able to suggest specific changes.
>
> The Automake manual has a chapter on how the distribution is built[1].
>
> A possible solution (not sure if it will work for you, you may have to
> experiment a bit):
>
>   - Prevent Automake from distributing any of the testsuite-related
>     flex source files (should be achievable using nodist_).
>   - Manually include only the files you actually want, e.g., by using
>     EXTRA_DIST.
See the attached files: test.l (dummy scanner, really) and a stub of configure.ac and Makefile.am that reproduce what I think should work.

Inlined for convenience of those reading as well:

test.l:
%%

configure.ac
AC_INIT([distribute flex without intermediate c files], 0.0.1, [hidden email], basketcase)
AC_CONFIG_SRCDIR([test.l])
AC_CONFIG_AUX_DIR([build-aux])
AM_INIT_AUTOMAKE([foreign parallel-tests ])
AC_CONFIG_MACRO_DIR([m4])
AC_PROG_CC
AM_PROG_CC_C_O

AM_PROG_LEX

AC_CONFIG_FILES(
Makefile)

AC_OUTPUT

and Makefile.am:

check_PROGRAMS = test

test_SOURCES = test.l
nodist_test_SOURCES = test.c

I had to manually create the m4/ directory, but then:

autoreconf -if && ./configure && make && make dist

produces a tar ball with test.c in it. despite the nodist_test_SOURCES line.

So what am I missing?


>
> Using a dist-hook may be helpful if EXTRA_DIST is not expressive enough.
>
> [1] https://gnu.org/software/automake/manual/automake.html#Dist
>
> Cheers,
>   Nick
>

--
Will Estes
[hidden email]

configure.ac (317 bytes) Download Attachment
test.l (4 bytes) Download Attachment
Makefile.am (78 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excluding intermediate flex test suite files from distribution?

Gavin Smith
On 12 November 2015 at 19:19, Will Estes <[hidden email]> wrote:

> and Makefile.am:
>
> check_PROGRAMS = test
>
> test_SOURCES = test.l
> nodist_test_SOURCES = test.c
>
> I had to manually create the m4/ directory, but then:
>
> autoreconf -if && ./configure && make && make dist
>
> produces a tar ball with test.c in it. despite the nodist_test_SOURCES line.
>
> So what am I missing?
>

I'd guess that nodit_test_SOURCES = test.c is wrong, because of this:

"You should never explicitly mention the intermediate (C or C++) file
in any SOURCES variable; only list the source file."

http://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html

I changed Makefile.am to

check_PROGRAMS = test

nodist_test_SOURCES = test.l
#nodist_test_SOURCES = test.c

then after "make dist":

$tar tf basketcase-0.0.1.tar.gz
basketcase-0.0.1/
basketcase-0.0.1/Makefile.in
basketcase-0.0.1/aclocal.m4
basketcase-0.0.1/configure
basketcase-0.0.1/Makefile.am
basketcase-0.0.1/configure.ac
basketcase-0.0.1/build-aux/
basketcase-0.0.1/build-aux/depcomp
basketcase-0.0.1/build-aux/install-sh
basketcase-0.0.1/build-aux/missing
basketcase-0.0.1/build-aux/ylwrap
basketcase-0.0.1/build-aux/compile

I guess that's not what you want, because test.l isn't distributed there.

Following Nick's suggestion to use dist-hook, the following appeared
to give good results:

check_PROGRAMS = test

test_SOURCES = test.l

dist-hook:
        rm -f ${distdir}/test.c

With that, test.l is distributed but test.c isn't.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excluding intermediate flex test suite files from distribution?

Will Estes
On Thursday, 12 November 2015,  9:41 pm +0000, Gavin Smith <[hidden email]> wrote:

> On 12 November 2015 at 19:19, Will Estes <[hidden email]> wrote:
> > and Makefile.am:
> >
> > check_PROGRAMS = test
> >
> > test_SOURCES = test.l
> > nodist_test_SOURCES = test.c
> >
> > I had to manually create the m4/ directory, but then:
> >
> > autoreconf -if && ./configure && make && make dist
> >
> > produces a tar ball with test.c in it. despite the nodist_test_SOURCES line.
> >
> > So what am I missing?
> >
>
> I'd guess that nodit_test_SOURCES = test.c is wrong, because of this:
>
> "You should never explicitly mention the intermediate (C or C++) file
> in any SOURCES variable; only list the source file."
>
> http://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html
>
> I changed Makefile.am to
>
> check_PROGRAMS = test
>
> nodist_test_SOURCES = test.l
> #nodist_test_SOURCES = test.c
>
> then after "make dist":
>
> $tar tf basketcase-0.0.1.tar.gz
> basketcase-0.0.1/
> basketcase-0.0.1/Makefile.in
> basketcase-0.0.1/aclocal.m4
> basketcase-0.0.1/configure
> basketcase-0.0.1/Makefile.am
> basketcase-0.0.1/configure.ac
> basketcase-0.0.1/build-aux/
> basketcase-0.0.1/build-aux/depcomp
> basketcase-0.0.1/build-aux/install-sh
> basketcase-0.0.1/build-aux/missing
> basketcase-0.0.1/build-aux/ylwrap
> basketcase-0.0.1/build-aux/compile
>
> I guess that's not what you want, because test.l isn't distributed there.
>
> Following Nick's suggestion to use dist-hook, the following appeared
> to give good results:
>
> check_PROGRAMS = test
>
> test_SOURCES = test.l
>
> dist-hook:
>         rm -f ${distdir}/test.c
>
> With that, test.l is distributed but test.c isn't.
>

Thanks. That does what I'm looking for and I can scale that up to the entire test suite with a bit of effort.

--
Will Estes
[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excluding intermediate flex test suite files from distribution?

Nick Bowler
On 11/12/15, Will Estes <[hidden email]> wrote:
> On Thursday, 12 November 2015,  9:41 pm +0000, Gavin Smith
> <[hidden email]> wrote:
>> On 12 November 2015 at 19:19, Will Estes <[hidden email]> wrote:
>> > and Makefile.am:
>> >
>> > check_PROGRAMS = test
>> >
>> > test_SOURCES = test.l
>> > nodist_test_SOURCES = test.c
[...]

>> I'd guess that nodit_test_SOURCES = test.c is wrong, because of this:
>>
>> "You should never explicitly mention the intermediate (C or C++) file
>> in any SOURCES variable; only list the source file."
>>
>> http://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html
>>
>> I changed Makefile.am to
>>
>> check_PROGRAMS = test
>>
>> nodist_test_SOURCES = test.l
>> #nodist_test_SOURCES = test.c
[...]

>> I guess that's not what you want, because test.l isn't distributed there.
>>
>> Following Nick's suggestion to use dist-hook, the following appeared
>> to give good results:
>>
>> check_PROGRAMS = test
>>
>> test_SOURCES = test.l
>>
>> dist-hook:
>>         rm -f ${distdir}/test.c
>>
>> With that, test.l is distributed but test.c isn't.
>>
>
> Thanks. That does what I'm looking for and I can scale that up to the entire
> test suite with a bit of effort.

This will probably work fine, but this has the potential problem that
test.c will be built every time you run 'make dist', just to be deleted
immediately.  This may or may not be a concern for you, but the likely
consequence is that lex will be required to run 'make dist', even though
it would otherwise not be required.

My original suggestion was to try something like this (not tested):

  EXTRA_DIST =

  nodist_test_SOURCES = test.l
  EXTRA_DIST += $(nodist_test_SOURCES)

Cheers,
  Nick

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excluding intermediate flex test suite files from distribution?

Will Estes
On Friday, 13 November 2015, 11:43 am -0500, Nick Bowler <[hidden email]> wrote:

> This will probably work fine, but this has the potential problem that
> test.c will be built every time you run 'make dist', just to be deleted
> immediately.  This may or may not be a concern for you, but the likely
> consequence is that lex will be required to run 'make dist', even though
> it would otherwise not be required.
>
> My original suggestion was to try something like this (not tested):

It is unfortunate, yes, but since automake already pulls in a requirement to have a lex on the build system, I can live with the slow down / extra disk I/O.

>
>   EXTRA_DIST =
>
>   nodist_test_SOURCES = test.l
>   EXTRA_DIST += $(nodist_test_SOURCES)

Thanks for the reminder. I haven't tested this, but the dist-hook approach works beautifully, given that I'm ok with the build and then delete that happens.

>
> Cheers,
>   Nick
>

--
Will Estes
[hidden email]

Loading...