[ANN] Leiningen 2.4.0

[Phil Hagelberg]

Greetings!

I'm pleased to announce the release of Leiningen 2.4.0, our most
significant release in quite some time. The star of the show is the new
`release` task, which can automate all the common steps behind the
release of a new version of a library.

The idea is that `lein release` will run a series of tasks you've
specified as release steps. This defaults to running the equivalent of
these commands:

* lein vcs assert-committed
* lein change version leiningen.release/bump-version release
* lein vcs commit
* lein vcs tag
* lein deploy
* lein change version leiningen.release/bump-version patch
* lein vcs commit
* lein vcs push

The `vcs` and `change` tasks are also new. The `vcs` subtasks above are
self-explanatory, but the `change` task allows for programmatic
modification of any given key in defproject using an arbitrary function,
without altering formatting or comments. Out of the box we ship a
function to bump version numbers, but you can call functions from
plugins, and we hope to add more built-in functions in future releases.

Note that the release task runs `lein deploy` rather than `lein deploy
clojars`; this is in order to prevent accidental deploys of private
artifacts to Clojars. In order to deploy an open source library to
Clojars it's necessary to add `:deploy-repositories {"releases" :clojars}`
to your project.clj file.

Other highlights include fixes to the search task to report index
download progress, improved multi-field queries, and the default to make
useful uberjars even when not AOTing a -main function.

A full list of significant changes:

* Allow aliases to splice in values from the project map. (Phil Hagelberg)
* Allow plugins to override built-in tasks. (Phil Hagelberg)
* Add `release` task for automating common release steps. (Wayne Warren, Chris Truter, Phil Hagelberg)
* Add `change` task for programmatc `project.clj` manipulation. (Chris Truter, Max Barnash)
* Abort when `defproject` contains duplicate keys. (Peter Garbers)
* Add `vcs` task to automate version control. (Phil Hagelberg, Wayne Warren)
* Automatically `clean` before `deploy` to avoid AOT in libraries. (Phil Hagelberg)
* Emit warnings to stderr. (Andy Chambers)
* Use `clojure.main` for uberjars that don't declare their own `:main`. (Phil Hagelberg)
* Allow templates to load from `:plugin-repositories`. (Phil Hagelberg)
* Fix a race condition on printing during dependency resolution. (Phil Hagelberg)
* Allow `new` templates to operate on existing directories with `--force` option. (Matthew Blair)
* Fix `search` task to allow queries on multiple fields. (Colin Jones)
* Fix a bug where errors in `run` task were mis-reported. (Gary Fredericks)
* Report download progress of search indices. (Matthew Blair)
* Protection from harmful `:clean-targets` settings. (Craig McDaniel)
* Faster loading of help text. (David Grayson, Ryan Mulligan)
* Add `LEIN_SILENT` option to suppress `*info*` output. (Phil Hagelberg)

As usual, for those who manually installed `lein upgrade` will pull in
the latest, and `lein upgrade 2.3.4` will back it down to the previous
version if you run into any issues.

Thanks to all the contributors who made this happen!

-Phil

ps. The Clojars artifacts for this release are still pending
investigation of some deploy issues, but they should be right around the
corner.

[Dave Tenny]

Will 'lein deploy clojars' still work as it does in prior versions if I
upgrade lein?

Post by Phil Hagelberg
Greetings!
I'm pleased to announce the release of Leiningen 2.4.0, our most
significant release in quite some time. The star of the show is the new
`release` task, which can automate all the common steps behind the
release of a new version of a library.
The idea is that `lein release` will run a series of tasks you've
specified as release steps. This defaults to running the equivalent of
* lein vcs assert-committed
* lein change version leiningen.release/bump-version release
* lein vcs commit
* lein vcs tag
* lein deploy
* lein change version leiningen.release/bump-version patch
* lein vcs commit
* lein vcs push
The `vcs` and `change` tasks are also new. The `vcs` subtasks above are
self-explanatory, but the `change` task allows for programmatic
modification of any given key in defproject using an arbitrary function,
without altering formatting or comments. Out of the box we ship a
function to bump version numbers, but you can call functions from
plugins, and we hope to add more built-in functions in future releases.
Note that the release task runs `lein deploy` rather than `lein deploy
clojars`; this is in order to prevent accidental deploys of private
artifacts to Clojars. In order to deploy an open source library to
Clojars it's necessary to add `:deploy-repositories {"releases" :clojars}`
to your project.clj file.
Other highlights include fixes to the search task to report index
download progress, improved multi-field queries, and the default to make
useful uberjars even when not AOTing a -main function.
* Allow aliases to splice in values from the project map. (Phil Hagelberg)
* Allow plugins to override built-in tasks. (Phil Hagelberg)
* Add `release` task for automating common release steps. (Wayne Warren,
Chris Truter, Phil Hagelberg)
* Add `change` task for programmatc `project.clj` manipulation. (Chris Truter, Max Barnash)
* Abort when `defproject` contains duplicate keys. (Peter Garbers)
* Add `vcs` task to automate version control. (Phil Hagelberg, Wayne Warren)
* Automatically `clean` before `deploy` to avoid AOT in libraries. (Phil Hagelberg)
* Emit warnings to stderr. (Andy Chambers)
* Use `clojure.main` for uberjars that don't declare their own `:main`. (Phil Hagelberg)
* Allow templates to load from `:plugin-repositories`. (Phil Hagelberg)
* Fix a race condition on printing during dependency resolution. (Phil Hagelberg)
* Allow `new` templates to operate on existing directories with `--force`
option. (Matthew Blair)
* Fix `search` task to allow queries on multiple fields. (Colin Jones)
* Fix a bug where errors in `run` task were mis-reported. (Gary Fredericks)
* Report download progress of search indices. (Matthew Blair)
* Protection from harmful `:clean-targets` settings. (Craig McDaniel)
* Faster loading of help text. (David Grayson, Ryan Mulligan)
* Add `LEIN_SILENT` option to suppress `*info*` output. (Phil Hagelberg)
As usual, for those who manually installed `lein upgrade` will pull in
the latest, and `lein upgrade 2.3.4` will back it down to the previous
version if you run into any issues.
Thanks to all the contributors who made this happen!
-Phil
ps. The Clojars artifacts for this release are still pending
investigation of some deploy issues, but they should be right around the
corner.

[Jean Niklas L'orange]

Post by Dave Tenny
Will 'lein deploy clojars' still work as it does in prior versions if I
upgrade lein?

Yes, the semantics of `lein release` in and by itself has not changed.

[Phil Hagelberg]

There's a bug in the 2.4.0 release that causes `lein help release` to
fail if you're running from an uberjarred lein release.

I've fixed it in master and will cut a release soon if no other bugs are
found. In the mean time it's safe to upgrade since the broken help
doesn't even exist in the old version.

Perform release tasks.
:release-tasks [["vcs" "assert-committed"]
["change" "version"
"leiningen.release/bump-version" "release"]
["vcs" "commit"]
["vcs" "tag"]
["deploy"]
["change" "version" "leiningen.release/bump-version"]
["vcs" "commit"]
["vcs" "push"]]
First change the version stored in project.clj, then commit that change, tag
this commit to with the release version indicated, deploy to the Maven release
repository, then change to the next snapshot version in project.clj, commit
that change, and push to the default remote version control repository.
A key point to note is that this default set of :release-tasks requires a clean
working directory as far as the current version control system is concerned.
This ensures that the `vcs commit` tasks will only save changes made to
project.clj made by the `change version` tasks.
This behavior can be overridden by setting :release-tasks a vector in which
every element is either a task name or a collection in which the first element
is a task name and the rest are arguments to that task.
The release task takes a single argument which should be one of :major,
:minor, :patch, :alpha, :beta, or :rc to indicate which version level to
bump. If none is given, it defaults to :patch.
Arguments: ([] [level])

Please let me know if you encounter any other issues.

-Phil

[Lee Spector]

We've been deploying our system to clojars and github with the update procedure (written for me by my student Tom Helmuth) shown at https://gist.github.com/thelmuth/1361411#merge-changes-into-master-lee-only.

This works but is a bit manual/cumbersome, and we were excited to read about the new commit/tag/deploy lein tasks that seem like they could simplify the process.

But trying to figure out the details consumed a couple of hours without a happy ending yet, with trouble involving getting GPG to work correctly.

We're probably not understanding something in the documentation, which seems to assume more knowledge than we have.

Is anybody able/willing to offer a little help on this?

I'll include below Tom's report to me on the specific issues he was having.

Thanks,

-Lee

The main trouble I ran into was getting GPG to work correctly, which lein uses to sign releases . After installing it (a bit of a headache), creating a keypair, and otherwise trying to follow the docs/tutorials I linked to you, I kept getting an error that I couldn't figure out how to fix when doing "lein deploy"
$ lein deploy
Could not decrypt credentials from /Users/user/.lein/credentials.clj.gpg
gpg: decrypt_message failed: eof
See `lein help gpg` for how to install gpg.
WARNING: please set :url in project.clj.
No credentials found for releases(did you mean `lein deploy clojars`?)
See `lein help deploying` for how to configure credentials to avoid prompts.
$ gpg --quiet --batch --decrypt ~/.lein/credentials.clj.gpg
gpg: gpg-agent is not available in this session
gpg: can't query passphrase in batch mode
gpg: Invalid passphrase; please try again ...
gpg: can't query passphrase in batch mode
gpg: Invalid passphrase; please try again ...
gpg: can't query passphrase in batch mode
gpg: decryption failed: secret key not available
What would really be great is a clear tutorial from beginning to end taking a new Clojure project on a machine with leiningen (but not GPG) to where you can use "lein release" successfully to deploy to Clojars. Parts of such a tutorial seem scattered in various other tutorials, but there are parts missing, like what needs to be added to the generic project.clj file, and how to get past the GPG bug I hit.
-Tom

[Phil Hagelberg]

The main trouble I ran into was getting GPG to work correctly, which
lein uses to sign releases . After installing it (a bit of a
headache), creating a keypair, and otherwise trying to follow the
docs/tutorials I linked to you, I kept getting an error that I
couldn't figure out how to fix when doing "lein deploy"
$ lein deploy
Could not decrypt credentials from /Users/user/.lein/credentials.clj.gpg
gpg: decrypt_message failed: eof

This is actually not a problem with deploying the jar; it's just about
not being able to read the encrypted credentials. You don't need
encrypted credentials in order to deploy; it merely streamlines things a
bit. Removing credentials.clj.gpg would at least allow the process to go
further if you don't mind entering your credentials each time you
deploy.

Granted there still might be issues with creating signatures,
(especially on Mac OS X, which seems to not have a very solid GPG setup)
but the problem described here is not about signatures.

-Phil

[Lee Spector]

Post by Phil Hagelberg

The main trouble I ran into was getting GPG to work correctly, which
lein uses to sign releases . After installing it (a bit of a
headache), creating a keypair, and otherwise trying to follow the
docs/tutorials I linked to you, I kept getting an error that I
couldn't figure out how to fix when doing "lein deploy"
$ lein deploy
Could not decrypt credentials from /Users/user/.lein/credentials.clj.gpg
gpg: decrypt_message failed: eof

This is actually not a problem with deploying the jar; it's just about
not being able to read the encrypted credentials. You don't need
encrypted credentials in order to deploy; it merely streamlines things a
bit. Removing credentials.clj.gpg would at least allow the process to go
further if you don't mind entering your credentials each time you
deploy.
Granted there still might be issues with creating signatures,
(especially on Mac OS X, which seems to not have a very solid GPG setup)
but the problem described here is not about signatures.

Thanks Phil.

Unfortunately we're still not clear about how to proceed... probably because we don't really know much about GPG or credentials.

I guess we will stick with our current manual deployment procedure for now, which works, although it's not as simple as it seems that "lein deploy" might be.

If anyone is inspired to document the full process for getting lein deploy to work in this situation then I'd love to see the results.

-Lee

[Phil Hagelberg]

Post by Lee Spector
Unfortunately we're still not clear about how to proceed... probably
because we don't really know much about GPG or credentials.
I guess we will stick with our current manual deployment procedure for
now, which works, although it's not as simple as it seems that "lein
deploy" might be.

Happy to help if I can, but I don't know what's wrong.

What happens when you try my suggestion of removing credentials.clj.gpg?

-Phil