Creation date: 2020-10-16
In this post I want to discuss the package manager of Julia including how to use it to install packages, update them, remove etc and how to create your own environment. In the end I show you the basics of creating a package.
Let's shortly discuss why we need a package manager and how to activate it.
It's often quite useful to not create everything from scratch but instead use packages that already exist to accomplish your goal.
Well I do write things from scratch from time to time to get a better feeling of how everything works and learn but even then there are some packages that I really don't want to miss.
One of those packages is Plots.jl which creates, who would have thought... plots. You don't want to code that yourself.
Okay how to install Plots?
julia> ] (@v1.5) pkg> add Plots
it's as easy as that. The
] brings you into the package mode. Other modes are discussed in the repl post.
If you want to check which packages are installed and in which version you can run:
(@v1.5) pkg> status
if you're only interested in a single package like: Which version of Plots do I have installed?
(@v1.5) pkg> status Plots Status `~/.julia/environments/v1.5/Project.toml` [91a5bcdd] Plots v1.6.12
some other useful commands are
(@v1.5) pkg> rm Plots
to remove the package and
(@v1.5) pkg> up
to update the packages.
Let's assume you're working on a single machine on different projects with different collaborators.
It might happen that you need different versions of a package for your own handful of projects. Additionally in your global environment you might always want to have the latest version.
First we create a folder
test in some folder on your computer.
Then start the julia repl and write
(@v1.5) pkg> activate . Activating new environment at `~/Julia/test/Project.toml` (test) pkg>
The last line shows that we are now in a new environment.
(test) pkg> status Status `~/Julia/test/Project.toml` (empty project)
If we need Plots in this environment we can write the same as before
(test) pkg> add Plots
and see that Plots
v1.6.12 is installed and it depends on a lot of other packages that depend on some others. All of this is saved in the
Manifest.toml in you new folder. You might want to have a look at that on your own.
Let's have a look at the
[deps] Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
We can see that we added a single package here. The
91a5bcdd-55d7-5caf-9e0b-520d859cae80 is the id of the package and can be ignored for now.
What happens if we want want to download an older version of Plots?
(test) pkg> add Plots@v1.1.0 Resolving package versions... Updating `~/Julia/test/Project.toml` [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0 Updating `~/Julia/test/Manifest.toml` [79e6a3ab] - Adapt v2.3.0 [e2d170a0] - DataValueInterfaces v1.0.0 [5ae413db] - EarCut_jll v2.1.5+0  - Formatting v0.4.1 [5c1252a2] - GeometryBasics v0.3.3 [c8e1da08] - IterTools v1.3.0  - IteratorInterfaceExtensions v1.0.0 [b964fa9f] - LaTeXStrings v1.2.0 [23fbe1c1] - Latexify v0.14.0 [1914dd2f] - MacroTools v0.5.5 [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0 [09ab397b] - StructArrays v0.4.4 [3783bdb8] - TableTraits v1.0.0 [bd369af6] - Tables v1.1.0
@and even a specific branch with
Now let's write some code inside a new file
using PkgVersion, Plots version = PkgVersion.Version(Plots) println("Version of Plots: $version")
as we are using
PkgVersion here we want to add that as well:
(test) pkg> add PkgVersion Resolving package versions... Updating `~/Julia/test/Project.toml` [eebad327] + PkgVersion v0.1.1 Updating `~/Julia/test/Manifest.toml` [eebad327] + PkgVersion v0.1.1
Okay. Let's restart the REPL i.e
julia> ] (@v1.5) pkg> activate . Activating environment at `~/Julia/test/Project.toml` (test) pkg> instantiate
We can have a look what it does with:
(test) pkg> ?instantiate instantiate [-v|--verbose] instantiate [-v|--verbose] [-m|--manifest] instantiate [-v|--verbose] [-p|--project] Download all the dependencies for the current project at the version given by the project's manifest. If no manifest exists or the --project option is given, resolve and download the dependencies compatible with the project.
now if we want to run our program:
(test) pkg> BACKSPACE TO RETURN TO julia> julia> include("test.jl") Version of Plots: 1.1.0
okay that worked well. The question is what happens when we close the Julia session? (Ctrl-D or
And then just run the program:
julia> include("test.jl") Version of Plots: 1.6.12
In this case we load the global version of Plots, which makes sense because we haven't changed to our new environment and are in the global environment as a default.
To recap: We need to use instantiate to make sure that we download the correct package versions for this environment.
Let's talk about a caveat with the current approach of using
] add Plots@v1.1.0
When we run
up we update
Plots again which might not be what we want.
In this case we can change the
[deps] PkgVersion = "eebad327-c553-4316-9ea0-9fa01ccd7688" Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" + [compat] + Plots = "= 1.1.0"
up again to downgrade Plots. (Yes
up can also downgrade)
You can read more about the
[compat] section in the Pkg Manual.
In some cases it makes sense to go a step further by creating your own package. It might be a hassle to always run
instantiate and maybe you want that you can instead write
using MyPackage which defines which packages it depends on and call your functions as you wish.
It has a different limitation though which I'll discuss at the end.
Let's create a simple
HelloWorld package which defines a method
greet() that greets the world.
First I go back to my projects folder where I previously created
(@v1.5) pkg> generate HelloWorld Generating project HelloWorld: HelloWorld/Project.toml HelloWorld/src/HelloWorld.jl
generate creates the
HelloWorld directory and
Project.toml and a
src folder with the
HelloWorld.jl file contains:
module HelloWorld greet() = print("Hello World!") end # module
Wow it anticipated what we wanted to do 😄
Let's have a look at the
Project.toml and change it to add
Plots to it.
name = "HelloWorld" uuid = "ff5fd036-50c3-45ef-84f2-3664a1a16633" authors = ["Ole Kröger <email@example.com>"] version = "0.1.0"
I think the
authors is from my git preferences but you can just change yours if it didn't work or add others to the list.
It automatically gave our package a
uuid like the Plots package as well as a version number.
We can now add:
[deps] PkgVersion = "eebad327-c553-4316-9ea0-9fa01ccd7688" Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" [compat] Plots = "= 1.1.0"
Then back to the REPL:
(@v1.5) pkg> dev HelloWorld [ Info: Resolving package identifier `HelloWorld` as a directory at `~/Julia/HelloWorld`. Path `HelloWorld` exists and looks like the correct package. Using existing path. Resolving package versions... Installed ArrayInterface ─── v2.13.6 Installed ExprTools ──────── v0.1.3 Installed LoweredCodeUtils ─ v1.2.4 Installed GLM ────────────── v1.3.11 Updating `~/.julia/environments/v1.5/Project.toml` [38e38edf] ↑ GLM v1.3.10 ⇒ v1.3.11 [ff5fd036] + HelloWorld v0.1.0 `../../../Julia/HelloWorld` [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0 [295af30f] ↑ Revise v3.1.0 ⇒ v3.1.5 Updating `~/.julia/environments/v1.5/Manifest.toml` [79e6a3ab] - Adapt v2.3.0 [4fba245c] ↑ ArrayInterface v2.13.5 ⇒ v2.13.6 [5ae413db] - EarCut_jll v2.1.5+0 [e2ba6199] ↑ ExprTools v0.1.2 ⇒ v0.1.3 [c87230d0] ↓ FFMPEG v0.4.0 ⇒ v0.3.0 [38e38edf] ↑ GLM v1.3.10 ⇒ v1.3.11 [28b8d3ca] ↓ GR v0.52.0 ⇒ v0.48.0 [5c1252a2] - GeometryBasics v0.3.3 [ff5fd036] + HelloWorld v0.1.0 `../../../Julia/HelloWorld` [692b3bcd] ↑ JLLWrappers v1.1.1 ⇒ v1.1.2 [6f1432cf] ↑ LoweredCodeUtils v1.2.3 ⇒ v1.2.4 [91a5bcdd] ↓ Plots v1.6.12 ⇒ v1.1.0 [295af30f] ↑ Revise v3.1.0 ⇒ v3.1.5 [09ab397b] - StructArrays v0.4.4
We can now see that Plots got downgraded in the global environment. It's therefore for a different use case. If a different package now needs a higher version of Plots we would get an error message.
devdoes the same as
addbut checks it our for development.
If you're interested in creating a package and not an environment you probably want to use PkgTemplates which creates a testing suite for you and makes it very easy to build docs and much more.
The above steps are if you want to create everything by yourself. 😉
You should now have a broad understanding of how the package manager works and how to create your own environment or package.
An environment is helpful for scripts that you share with some colleagues or that are just for your own and depend on specific versions of some packages. In contrast a package makes sense if you want to share it with the wider world and support a wide range and up to date versions of the dependencies.
The next part in the series is about debugging.
Thanks for reading and special thanks to my 10 patrons!
List of patrons paying more than 4$ per month:
Currently I get more than 60$ per month using Patreon and PayPal.
For a donation of a single dollar per month you get early access to these posts. Your support will also increase the time I can spend writing these to increase the quality as well as posting more. If you want to get some help for your own project you might want to check out my mentoring post.
I'll keep you updated on Twitter OpenSourcES.