Bug 11830 - Document disk space requirements for git install on Ubuntu
Summary: Document disk space requirements for git install on Ubuntu
Status: CLOSED INVALID
Alias: None
Product: Koha
Classification: Unclassified
Component: Installation and upgrade (command-line installer) (show other bugs)
Version: Main
Hardware: All Linux
: P5 - low minor
Assignee: Galen Charlton
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-02-24 15:25 UTC by Dan Scott
Modified: 2018-06-04 20:10 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: String patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Adds a HARDWARE REQUIREMENTS section to the INSTALL.ubuntu file (1.76 KB, patch)
2014-02-24 15:25 UTC, Dan Scott
Details | Diff | Splinter Review
Bug 11830 - Missing Disk Space recommendation for installation (1.56 KB, patch)
2014-02-25 17:36 UTC, Mark Tompsett
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Dan Scott 2014-02-24 15:25:16 UTC
Created attachment 25587 [details] [review]
Adds a HARDWARE REQUIREMENTS section to the INSTALL.ubuntu file

When I installed Koha for the first time on Ubuntu, I provisioned a VM with what I assumed would be a generous 4GB of disk space.

This turned out to be a very bad assumption due to the 2.3GB that a full git clone of the Koha repository requires. The symptoms of running out of disk space were sadly not immediately apparent and caused much head scratching.

Therefore, in INSTALL.ubuntu, include an upfront note about hardware requirements in the case of a git install, in the hopes that others will not end up experiencing a similar fate :)
Comment 1 Mark Tompsett 2014-02-25 17:18:08 UTC
I agree with the idea. However, you forgot to account for data size. The sizes you suggest are far too small to actually import and fully reindex a small production system set of data. Please change to 20GB, and I'll sign off. ;)
Comment 2 Mark Tompsett 2014-02-25 17:27:22 UTC
Last I checked a full clone of git was just over 1GB. A bare install of Ubuntu is let's say 2GB. I am using a 15GB VM, and I only have 2.7GB left. My data is about 45K records. Also, it doesn't matter if it is a VM or not VM.
Comment 3 Mark Tompsett 2014-02-25 17:36:44 UTC
Created attachment 25622 [details] [review]
Bug 11830 - Missing Disk Space recommendation for installation

This adds a section which recommends at least 20GB disk space
to the INSTALL.ubuntu file.
Comment 4 Mark Tompsett 2014-02-25 17:38:05 UTC
I've put up a revision. Feel free to sign it off, Dan.
Comment 5 Dan Scott 2014-02-26 07:06:26 UTC
M. Tompsett - I didn't forget to account for data size. As far as I know nobody recommends running Koha from git for production purposes; my assumption is that someone running from git is using it for development purposes. In which case most developers are going to be using a small data set, or will provision according to the size of their data.

What I'm trying to establish is a reasonable minimal baseline.

After "git clean -f" and "git reset --hard HEAD", "du -hs kohaclone" shows me a disk space usage of 1.9 GB.

After running "perl Makefile.PL" and "make", "du -hs kohaclone" shows me a disk space usage of 2.3 GB.

After running "make install", "du -hs /home/koha/koha-dev" then shows me disk space usage of 401 MB.

Thus, a minimal git install requires 2.7 GB of free space... rounding up to the 3 GB I recommended in my original patch seems reasonable.

On my 6.0 GB Ubuntu VM, here's what "df -h" shows after a minimal install and a load of approximately 1000 MARC records:

Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/koha--vg-root  5.6G  4.9G  413M  93% /
udev                       992M  4.0K  992M   1% /dev
tmpfs                      401M  276K  401M   1% /run
none                       5.0M     0  5.0M   0% /run/lock
none                      1002M     0 1002M   0% /run/shm
/dev/vda1                  228M   78M  138M  37% /boot

I stand by my original patch as-is, as it uses the terms "at least" to target the minimal case for a git install and for the minimal VM size.
Comment 6 Mark Tompsett 2014-02-26 17:22:58 UTC
Some people may actually want to load real data for testing. Larger is better.
Comment 7 Dan Scott 2014-02-26 17:50:00 UTC
(In reply to M. Tompsett from comment #6)
> Some people may actually want to load real data for testing. Larger is
> better.

Yes, but those people are going to know how large their data is and can determine how much disk space they will need on top of the documented minimum requirements.

If you want to provide a metric like "If you have 1 GB of MARC binary records, you should provision 3 GB of space to allow for indexing, etc", then I think that would be a helpful addition.
Comment 8 Galen Charlton 2014-02-26 17:57:08 UTC
To reflect the state of affairs correctly, I'm setting this to in-discussion and un-obsoleting Dan's patch.

Mark: please do not automatically obsolete the original patch when you're counter-patching.

As I've expressed on koha-devel, I believe that a better place for system specs, including for devs, is on the wiki or main website.

As far as the two patches are concerned, Dan's numbers add up.  I agree that a metric for estimating disk space per 1K bib records would be useful.
Comment 9 Galen Charlton 2014-02-26 18:03:53 UTC
(In reply to Dan Scott from comment #7)
> If you want to provide a metric like "If you have 1 GB of MARC binary
> records, you should provision 3 GB of space to allow for indexing, etc",
> then I think that would be a helpful addition.

One number that's reasonably conservative for small to medium-sized databases: 55M of disk per 1,000 MARC21 bib records for Zebra and MySQL storage.
Comment 10 Galen Charlton 2014-02-26 18:08:23 UTC
Final comment for now -- whatever form the specs take, there should be a clear distinction between specs for devs and specs for production, with some padding for production.

This is because while a lot of small Koha database could in fact reasonably live on a system with only 10G of disk -- I've seen far too many Koha setups where insufficient attention is paid to monitoring disk usage to want the official community recommendation to be too low.

In other words, I'm fine with Dan's suggestion of minimum 6G for development setups, but prefer Mark's suggestion of 20G for production usage.
Comment 11 Dan Scott 2014-02-26 18:10:43 UTC
(In reply to Galen Charlton from comment #10)
> As I've expressed on koha-devel, I believe that a better place for system
> specs, including for devs, is on the wiki or main website.

That sounds good to me, as long as there's a clear pointer from the INSTALL.* files to the appropriate URL for hardware requirements.
Comment 12 Mark Tompsett 2014-02-26 18:31:10 UTC
Okay. How about here:
http://koha-community.org/download-koha/
We could add some statement to the point list like:
* Hard drive space recommendations include >=6 GB for a minimal development install, >= 20GB for a production install

And then add that link to the USEFUL REFERENCE LINKS section like:

Download (requirements, etc.):
http://koha-community.org/download-koha/

Perhaps even include a link to a wiki page which flushes out disk space and memory suggestions?
Comment 13 Magnus Enger 2017-08-01 06:17:52 UTC
Bug 17626 removed the INSTALL.* files, so this bug is no longer relevant.