Setting default repositories for new projects

[bill robertson]

Hello,

I've been digging through search results on leiningen for a way to set
default repositories. I first ran into a discussion on init.clj, which
turns out to be unsupported in lein 2. The discussion on migrating from 1.x
to 2.x suggests to put things that used to be in init.clj into the user
profile.

However, adding repositories to user profile invokes a repeatability
warning. Which makes sense. I could leave it as is, but I really dislike
telling people to ignore warnings. It trains you to do the wrong thing. I
would not like to add the repositories stanza to each project.clj file by
hand either because it would be error prone.

Is there an easier way to get this in every project.clj file w/o warnings
and/or a lot of manual labor?

Thanks,
Bill Robertson

[Phil Hagelberg]

bill robertson writes:

> However, adding repositories to user profile invokes a repeatability
> warning. Which makes sense. I could leave it as is, but I really dislike
> telling people to ignore warnings. It trains you to do the wrong thing. I
> would not like to add the repositories stanza to each project.clj file by
> hand either because it would be error prone.

What is the motivation for wanting this?

One of the core values of Leiningen is that every project must be
repeatably buildable from nothing but the source and access to remote
(hopefully immutable) dependency repositories. Adding inputs to the
process from other places can look convenient in the short term, but
like dynamic binding it introduces subtle, difficult-to-trace problems.

-Phil

[bill robertson]

Hello Phil,

Thanks for the reply.

> What is the motivation for wanting this?

It's not 100% clear what you mean. If you mean do I want a feature change
in lein, then no I'm not asking for that.

If you're asking why we need to make sure we're pointing to our internal
repository rather than public ones, those reasons are business related. We
can not publish artifacts to a public repository. So a build that is not
pointing to that repo should simply fail. Although I can see how that would
be an issue if the location of the repository changed.

Given our need to get work done today, in our environment as it exists, I
would prefer to make it easy for new projects to be correctly configured
w/o having to remember to modify each one by copying/pasting the same
stanza into each project.clj file. Think about the headache that would
introduce if the location of the repository changed as compared to if it
were configured once? Seems like a DRY issue to me.

And, like I said earlier about warnings. I think they're a less than ideal
choice, because they simply train people to ignore problems. -Werror FTW ;-)

If there isn't really a way to do this currently and you don't plan to add
one, then that's fine. However, if you have advice or have seen how others
have approached this situation then I'd be happy to hear it.

Thanks,
Bill Robertson



On Tue, Mar 4, 2014 at 5:46 PM, Phil Hagelberg <phil-Tr7DW4wtY2wdnm+***@public.gmane.org> wrote:

>
> bill robertson writes:
>
> > However, adding repositories to user profile invokes a repeatability
> > warning. Which makes sense. I could leave it as is, but I really dislike
> > telling people to ignore warnings. It trains you to do the wrong thing. I
> > would not like to add the repositories stanza to each project.clj file by
> > hand either because it would be error prone.
>
> What is the motivation for wanting this?
>
> One of the core values of Leiningen is that every project must be
> repeatably buildable from nothing but the source and access to remote
> (hopefully immutable) dependency repositories. Adding inputs to the
> process from other places can look convenient in the short term, but
> like dynamic binding it introduces subtle, difficult-to-trace problems.
>
> -Phil
>

[David Powell]

I don't currently have an intranet leiningen repo set up at work, but I can
imagine similar issues. While clojars.org isn't likely to suddenly be
renamed to a different host, in an intranet environment servers do get
moved, servers from different departments might get merged etc. Burning
not-very-permanent names into project.clj is problematic because
project.clj goes into source control and then its history is immutable. I
can change project.clj if a server moves, but then I'm going to have to
manually patch up the file if I need to reproduce an historic build. In
effect the history of project.clj is independent of the history of the
context of repo locations.

Putting repos in project.clj is sensible if the repos can be guaranteed to
be permanent and immutable. If the reality is that repo names are
contextual or not permanent, then I think the user profile is a better
place for them, and in that situation it is legit to suppress the warning.



On Wed, Mar 5, 2014 at 2:29 PM, bill robertson <billrobertson42-***@public.gmane.org>wrote:

> Hello Phil,
>
> Thanks for the reply.
>
> > What is the motivation for wanting this?
>
> It's not 100% clear what you mean. If you mean do I want a feature change
> in lein, then no I'm not asking for that.
>
> If you're asking why we need to make sure we're pointing to our internal
> repository rather than public ones, those reasons are business related. We
> can not publish artifacts to a public repository. So a build that is not
> pointing to that repo should simply fail. Although I can see how that would
> be an issue if the location of the repository changed.
>
> Given our need to get work done today, in our environment as it exists, I
> would prefer to make it easy for new projects to be correctly configured
> w/o having to remember to modify each one by copying/pasting the same
> stanza into each project.clj file. Think about the headache that would
> introduce if the location of the repository changed as compared to if it
> were configured once? Seems like a DRY issue to me.
>
> And, like I said earlier about warnings. I think they're a less than ideal
> choice, because they simply train people to ignore problems. -Werror FTW
> ;-)
>
> If there isn't really a way to do this currently and you don't plan to add
> one, then that's fine. However, if you have advice or have seen how others
> have approached this situation then I'd be happy to hear it.
>
> Thanks,
> Bill Robertson
>
>
>
> On Tue, Mar 4, 2014 at 5:46 PM, Phil Hagelberg <phil-Tr7DW4wtY2wdnm+***@public.gmane.org> wrote:
>
>>
>> bill robertson writes:
>>
>> > However, adding repositories to user profile invokes a repeatability
>> > warning. Which makes sense. I could leave it as is, but I really dislike
>> > telling people to ignore warnings. It trains you to do the wrong thing.
>> I
>> > would not like to add the repositories stanza to each project.clj file
>> by
>> > hand either because it would be error prone.
>>
>> What is the motivation for wanting this?
>>
>> One of the core values of Leiningen is that every project must be
>> repeatably buildable from nothing but the source and access to remote
>> (hopefully immutable) dependency repositories. Adding inputs to the
>> process from other places can look convenient in the short term, but
>> like dynamic binding it introduces subtle, difficult-to-trace problems.
>>
>> -Phil
>>
>
>

[Sean Corfield]

But you have to update the project.clj file in each new project already to add dependencies (to the things that are in your private repo) so you're already "copying/pasting the same stanza" aren't you?

And how often are you creating new projects?

(just trying to understand the amount of work you think you'd save here)

Sean

On Mar 5, 2014, at 6:29 AM, bill robertson <billrobertson42-***@public.gmane.org> wrote:
> Given our need to get work done today, in our environment as it exists, I would prefer to make it easy for new projects to be correctly configured w/o having to remember to modify each one by copying/pasting the same stanza into each project.clj file. Think about the headache that would introduce if the location of the repository changed as compared to if it were configured once? Seems like a DRY issue to me.
>
> And, like I said earlier about warnings. I think they're a less than ideal choice, because they simply train people to ignore problems. -Werror FTW ;-)
>
> If there isn't really a way to do this currently and you don't plan to add one, then that's fine. However, if you have advice or have seen how others have approached this situation then I'd be happy to hear it.
>
> Thanks,
> Bill Robertson
>
>
>
> On Tue, Mar 4, 2014 at 5:46 PM, Phil Hagelberg <phil-Tr7DW4wtY2wdnm+***@public.gmane.org> wrote:
>
> bill robertson writes:
>
> > However, adding repositories to user profile invokes a repeatability
> > warning. Which makes sense. I could leave it as is, but I really dislike
> > telling people to ignore warnings. It trains you to do the wrong thing. I
> > would not like to add the repositories stanza to each project.clj file by
> > hand either because it would be error prone.
>
> What is the motivation for wanting this?
>
> One of the core values of Leiningen is that every project must be
> repeatably buildable from nothing but the source and access to remote
> (hopefully immutable) dependency repositories. Adding inputs to the
> process from other places can look convenient in the short term, but
> like dynamic binding it introduces subtle, difficult-to-trace problems.
>
> -Phil

[Kurtis Mullins]

I'm no lein expert by any measure -- just throwing this out there ... but
maybe this can be solved with devops or a project template (is that even in
the right ballpark?)


On Wed, Mar 5, 2014 at 12:53 PM, Sean Corfield <sean-***@public.gmane.org> wrote:

> But you have to update the project.clj file in each new project already to
> add dependencies (to the things that are in your private repo) so you're
> already "copying/pasting the same stanza" aren't you?
>
> And how often are you creating new projects?
>
> (just trying to understand the amount of work you think you'd save here)
>
> Sean
>
> On Mar 5, 2014, at 6:29 AM, bill robertson <billrobertson42-***@public.gmane.org>
> wrote:
>
> Given our need to get work done today, in our environment as it exists, I
> would prefer to make it easy for new projects to be correctly configured
> w/o having to remember to modify each one by copying/pastin g the same
> stanza into each project.clj file. Think about the headache that would
> introduce if the location of the repository changed as compared to if it
> were configured once? Seems like a DRY issue to me.
>
> And, like I said earlier about warnings. I think they're a less than ideal
> choice, because they simply train people to ignore problems. -Werror FTW
> ;-)
>
> If there isn't really a way to do this currently and you don't plan to add
> one, then that's fine. However, if you have advice or have seen how others
> have approached this situation then I'd be happy to hear it.
>
> Thanks,
> Bill Robertson
>
>
>
> On Tue, Mar 4, 2014 at 5:46 PM, Phil Hagelberg <phil-Tr7DW4wtY2wdnm+***@public.gmane.org> wrote:
>
>>
>> bill robertson writes:
>>
>> > However, adding repositories to user profile invokes a repeatability
>> > warning. Which makes sense. I could leave it as is, but I really dislike
>> > telling people to ignore warnings. It trains you to do the wrong thing.
>> I
>> > would not like to add the repositories stanza to each project.clj file
>> by
>> > hand either because it would be error prone.
>>
>> What is the motivation for wanting this?
>>
>> One of the core values of Leiningen is that every project must be
>> repeatably buildable from nothing but the source and access to remote
>> (hopefully immutable) dependency repositories. Adding inputs to the
>> process from other places can look convenient in the short term, but
>> like dynamic binding it introduces subtle, difficult-to-trace problems.
>>
>> -Phil
>>
>
>

[Phil Hagelberg]

bill robertson writes:

> Given our need to get work done today, in our environment as it exists, I
> would prefer to make it easy for new projects to be correctly configured
> w/o having to remember to modify each one by copying/pasting the same
> stanza into each project.clj file. Think about the headache that would
> introduce if the location of the repository changed as compared to if it
> were configured once? Seems like a DRY issue to me.

One way to do this is using project middleware. You can publish a
leiningen plugin containing a function that takes a project map as its
argument which calls update-in on :repositories to add the value you
want. This moves the definition of the repository URL to a symbolic
:plugins reference rather than hard-coding it. However, you still have
the issue of immutability--the version number will change in the future,
so you have to depend on the "RELEASE" version, which checks for a new
version every 24 hours. But the project checkout has everything that's
necessary for it to run.

I would recommend against the middleware approach as it leaves too much
as implicit and has a lot of moving parts. It's much better to just
include the :repositories value in the project.clj file.

If the repository moves in the future, it's easy to work around this
problem by adding :mirrors to your user profile. This is a way of
telling Leiningen that you have a better way of accessing a given
repository which for whatever reason should be preferred. That reason is
usually better network locality, but it could just as easily be that the
repository has moved.

-Phil

[Jean Niklas L'orange]

Hello Bill,

On 4 March 2014 23:32, bill robertson <billrobertson42-***@public.gmane.org> wrote:
>
> Is there an easier way to get this in every project.clj file w/o warnings
> and/or a lot of manual labor?
>

I think you can do this through profiles.d and some dereferencing:

In `~/.lein/profiles.d/my-company-defaults.clj`, insert the default
repositories. Then, in the project.clj, add in the following:

(def company-repos
(get-in (leiningen.core.user/profiles-d-profiles)
[:my-company-defaults :repositories]
^:replace [])) ;; Ensure failure if the profile lookup fails.

(defproject my-project ...
:repositories ~company-repos
...)

Of course, instead of changing `intrarepo.mycompany.com` in all the
different `project.clj`'s, you should change the DNS entry of `
intrarepo.mycompany.com`. As far as I know, that's the rationale behind
having a DNS entry in the first place, isn't it?

I would recommend to make an internal template for this, regardless of the
course you take. That removes the small overhead related to each new
project your company create.

--
Regards,
Jean Niklas L'orange

[bill robertson]

That's an interesting approach. I've been thinking about the template route
as well.

Thanks for the feedback.

-Bill


On Wed, Mar 5, 2014 at 1:56 PM, Jean Niklas L'orange
<jeannikl-DgnzNwtltm5Wk0Htik3J/***@public.gmane.org>wrote:

> Hello Bill,
>
> On 4 March 2014 23:32, bill robertson <billrobertson42-***@public.gmane.org> wrote:
>>
>> Is there an easier way to get this in every project.clj file w/o warnings
>> and/or a lot of manual labor?
>>
>
> I think you can do this through profiles.d and some dereferencing:
>
> In `~/.lein/profiles.d/my-company-defaults.clj`, insert the default
> repositories. Then, in the project.clj, add in the following:
>
> (def company-repos
> (get-in (leiningen.core.user/profiles-d-profiles)
> [:my-company-defaults :repositories]
> ^:replace [])) ;; Ensure failure if the profile lookup fails.
>
> (defproject my-project ...
> :repositories ~company-repos
> ...)
>
> Of course, instead of changing `intrarepo.mycompany.com` in all the
> different `project.clj`'s, you should change the DNS entry of `
> intrarepo.mycompany.com`. As far as I know, that's the rationale behind
> having a DNS entry in the first place, isn't it?
>
> I would recommend to make an internal template for this, regardless of the
> course you take. That removes the small overhead related to each new
> project your company create.
>
> --
> Regards,
> Jean Niklas L'orange
>