The Julia Package Manager

Welcome back to a new Julia Basics post. In the last two posts we discussed multiple dispatch and how to work with the repl and Revise.jl.

In this post I want to discuss the package manager of Julia including how to use it to install packages, update them, remove etc and how to create your own environment. In the end I show you the basics of creating a package.

The package manager

Let's shortly discuss why we need a package manager and how to activate it.

It's often quite useful to not create everything from scratch but instead use packages that already exist to accomplish your goal.

Well I do write things from scratch from time to time to get a better feeling of how everything works and learn but even then there are some packages that I really don't want to miss.

One of those packages is Plots.jl which creates, who would have thought... plots. You don't want to code that yourself.

Okay how to install Plots?

julia> ]
(@v1.5) pkg> add Plots

it's as easy as that. The ] brings you into the package mode. Other modes are discussed in the repl post.

If you want to check which packages are installed and in which version you can run:

(@v1.5) pkg> status

if you're only interested in a single package like: Which version of Plots do I have installed?

(@v1.5) pkg> status Plots
Status `~/.julia/environments/v1.5/Project.toml`
  [91a5bcdd] Plots v1.6.12

some other useful commands are

(@v1.5) pkg> rm Plots

to remove the package and

(@v1.5) pkg> up

to update the packages.

Environments

Let's assume you're working on a single machine on different projects with different collaborators.

It might happen that you need different versions of a package for your own handful of projects. Additionally in your global environment you might always want to have the latest version.

First we create a folder test in some folder on your computer.

Then start the julia repl and write

(@v1.5) pkg> activate .
 Activating new environment at `~/Julia/test/Project.toml`

(test) pkg>

The last line shows that we are now in a new environment.

(test) pkg> status
Status `~/Julia/test/Project.toml` (empty project)

If we need Plots in this environment we can write the same as before

(test) pkg> add Plots

and see that Plots v1.6.12 is installed and it depends on a lot of other packages that depend on some others. All of this is saved in the Manifest.toml in you new folder. You might want to have a look at that on your own.

Let's have a look at the Project.toml:

[deps]
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"

We can see that we added a single package here. The 91a5bcdd-55d7-5caf-9e0b-520d859cae80 is the id of the package and can be ignored for now.

What happens if we want want to download an older version of Plots?

(test) pkg> add Plots@v1.1.0
  Resolving package versions...
Updating `~/Julia/test/Project.toml`
  [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0
Updating `~/Julia/test/Manifest.toml`
  [79e6a3ab] - Adapt v2.3.0
  [e2d170a0] - DataValueInterfaces v1.0.0
  [5ae413db] - EarCut_jll v2.1.5+0
  [59287772] - Formatting v0.4.1
  [5c1252a2] - GeometryBasics v0.3.3
  [c8e1da08] - IterTools v1.3.0
  [82899510] - IteratorInterfaceExtensions v1.0.0
  [b964fa9f] - LaTeXStrings v1.2.0
  [23fbe1c1] - Latexify v0.14.0
  [1914dd2f] - MacroTools v0.5.5
  [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0
  [09ab397b] - StructArrays v0.4.4
  [3783bdb8] - TableTraits v1.0.0
  [bd369af6] - Tables v1.1.0

Now let's write some code inside a new file test.jl:

using PkgVersion, Plots

version = PkgVersion.Version(Plots)
println("Version of Plots: $version")

as we are using PkgVersion here we want to add that as well:

(test) pkg> add PkgVersion
  Resolving package versions...
Updating `~/Julia/test/Project.toml`
  [eebad327] + PkgVersion v0.1.1
Updating `~/Julia/test/Manifest.toml`
  [eebad327] + PkgVersion v0.1.1

Okay. Let's restart the REPL i.e CTRL-D and julia

julia> ]
(@v1.5) pkg> activate .
 Activating environment at `~/Julia/test/Project.toml`

(test) pkg> instantiate

We can have a look what it does with:

(test) pkg> ?instantiate

  instantiate [-v|--verbose]
  instantiate [-v|--verbose] [-m|--manifest]
  instantiate [-v|--verbose] [-p|--project]

  Download all the dependencies for the current project at the version given by the project's manifest. If no manifest exists or the --project option is given, resolve and download the
  dependencies compatible with the project.

now if we want to run our program:

(test) pkg>  BACKSPACE TO RETURN TO julia>
julia> include("test.jl")
Version of Plots: 1.1.0

okay that worked well. The question is what happens when we close the Julia session? (Ctrl-D or exit())

And then just run the program:

julia> include("test.jl")
Version of Plots: 1.6.12

In this case we load the global version of Plots, which makes sense because we haven't changed to our new environment and are in the global environment as a default.

To recap: We need to use instantiate to make sure that we download the correct package versions for this environment.

Let's talk about a caveat with the current approach of using ] add Plots@v1.1.0

When we run activate and up we update Plots again which might not be what we want.

In this case we can change the Project.toml

[deps]
PkgVersion = "eebad327-c553-4316-9ea0-9fa01ccd7688"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"

+ [compat]
+ Plots = "= 1.1.0"

and run up again to downgrade Plots. (Yes up can also downgrade)

You can read more about the [compat] section in the Pkg Manual.

Generating your own package

In some cases it makes sense to go a step further by creating your own package. It might be a hassle to always run instantiate and maybe you want that you can instead write using MyPackage which defines which packages it depends on and call your functions as you wish.

It has a different limitation though which I'll discuss at the end.

Let's create a simple HelloWorld package which defines a method greet() that greets the world.

First I go back to my projects folder where I previously created test.

(@v1.5) pkg> generate HelloWorld
 Generating  project HelloWorld:
    HelloWorld/Project.toml
    HelloWorld/src/HelloWorld.jl

generate creates the HelloWorld directory and Project.toml and a src folder with the HelloWorld.jl file.

The HelloWorld.jl file contains:

module HelloWorld

greet() = print("Hello World!")

end # module

Wow it anticipated what we wanted to do 😄

Let's have a look at the Project.toml and change it to add PkgVersion and Plots to it.

name = "HelloWorld"
uuid = "ff5fd036-50c3-45ef-84f2-3664a1a16633"
authors = ["Ole Kröger <o.kroeger@opensourc.es>"]
version = "0.1.0"

I think the authors is from my git preferences but you can just change yours if it didn't work or add others to the list.

It automatically gave our package a uuid like the Plots package as well as a version number.

We can now add:

[deps]
PkgVersion = "eebad327-c553-4316-9ea0-9fa01ccd7688"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"

[compat]
Plots = "= 1.1.0"

Then back to the REPL:

(@v1.5) pkg> dev HelloWorld
[ Info: Resolving package identifier `HelloWorld` as a directory at `~/Julia/HelloWorld`.
Path `HelloWorld` exists and looks like the correct package. Using existing path.
  Resolving package versions...
  Installed ArrayInterface ─── v2.13.6
  Installed ExprTools ──────── v0.1.3
  Installed LoweredCodeUtils ─ v1.2.4
  Installed GLM ────────────── v1.3.11
Updating `~/.julia/environments/v1.5/Project.toml`
  [38e38edf] ↑ GLM v1.3.10 ⇒ v1.3.11
  [ff5fd036] + HelloWorld v0.1.0 `../../../Julia/HelloWorld`
  [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0
  [295af30f] ↑ Revise v3.1.0 ⇒ v3.1.5
Updating `~/.julia/environments/v1.5/Manifest.toml`
  [79e6a3ab] - Adapt v2.3.0
  [4fba245c] ↑ ArrayInterface v2.13.5 ⇒ v2.13.6
  [5ae413db] - EarCut_jll v2.1.5+0
  [e2ba6199] ↑ ExprTools v0.1.2 ⇒ v0.1.3
  [c87230d0] ↓ FFMPEG v0.4.0 ⇒ v0.3.0
  [38e38edf] ↑ GLM v1.3.10 ⇒ v1.3.11
  [28b8d3ca] ↓ GR v0.52.0 ⇒ v0.48.0
  [5c1252a2] - GeometryBasics v0.3.3
  [ff5fd036] + HelloWorld v0.1.0 `../../../Julia/HelloWorld`
  [692b3bcd] ↑ JLLWrappers v1.1.1 ⇒ v1.1.2
  [6f1432cf] ↑ LoweredCodeUtils v1.2.3 ⇒ v1.2.4
  [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0
  [295af30f] ↑ Revise v3.1.0 ⇒ v3.1.5
  [09ab397b] - StructArrays v0.4.4

We can now see that Plots got downgraded in the global environment. It's therefore for a different use case. If a different package now needs a higher version of Plots we would get an error message.

If you're interested in creating a package and not an environment you probably want to use PkgTemplates which creates a testing suite for you and makes it very easy to build docs and much more.

The above steps are if you want to create everything by yourself. 😉

Conclusion

You should now have a broad understanding of how the package manager works and how to create your own environment or package.

An environment is helpful for scripts that you share with some colleagues or that are just for your own and depend on specific versions of some packages. In contrast a package makes sense if you want to share it with the wider world and support a wide range and up to date versions of the dependencies.

The next part in the series is about debugging.

For all posts in this series please check the list in the sidebar.

Thanks for reading and special thanks to my 10 patrons!

List of patrons paying more than 4$ per month:

• Site Wang

• Gurvesh Sanghera

• Szymon Bęczkowski

• DShiu

Currently I get more than 60$ per month using Patreon and PayPal.

For a donation of a single dollar per month you get early access to these posts. Your support will also increase the time I can spend writing these to increase the quality as well as posting more. If you want to get some help for your own project you might want to check out my mentoring post.

I'll keep you updated on Twitter OpenSourcES.

Want to be updated? Consider subscribing on Patreon for free