Bug 33115

Summary: koha-dump: create option to include htdocs for OPAC and Intranet sites
Product: Koha Reporter: David Liddle <david.liddle>
Component: Command-line UtilitiesAssignee: Bugs List <koha-bugs>
Status: CLOSED INVALID QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: dcook, robin
Version: 22.05   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description David Liddle 2023-03-02 10:45:37 UTC
Hello! I have found recently that the results of koha-dump do not include the files within the htdocs directories of the OPAC and Intranet sites under /usr/share/koha. These directories would typically contain files added or altered to customize the look and appearance of each. It would be helpful perhaps to have a switch such as --include-htdocs so that the files can be preserved (and restored with koha-restore) in one action.

(However, if there is a different method /directory preferred for such customizations, then I respectfully yield to the wisdom of the development team.)
Comment 1 David Cook 2023-03-02 22:14:25 UTC
What files are you editing in those directories?
Comment 2 David Liddle 2023-03-03 11:16:41 UTC
Logos and custom fonts were added to the root htdocs directory for both sites, and there is a modified XSLT stylesheet in use for the intranet site. All of these are referenced by system preferences that would be included in a restore.

If there is a better location for such files, such that they would be dumped and restored by the existing processes, then I would happily alter our practice.
Comment 3 David Cook 2023-03-05 22:56:32 UTC
(In reply to David Liddle from comment #2)
> Logos and custom fonts were added to the root htdocs directory for both
> sites, and there is a modified XSLT stylesheet in use for the intranet site.
> All of these are referenced by system preferences that would be included in
> a restore.
> 
> If there is a better location for such files, such that they would be dumped
> and restored by the existing processes, then I would happily alter our
> practice.

You could create a /var/lib/koha/<instancename>/web folder and then use an Alias in Apache in /etc/koha/apache-shared-opac.conf like the following:

Alias /web "/var/lib/koha/${instance}/web"

You may also need a directive like the following depending on your security:

<Directory "/var/lib/koha/${instance}/web">
    Require all granted
</Directory>

The /var/lib/koha/<instancename> directory gets automatically backed up by koha-dump. 

--

It does require customization of the Apache config, so maybe we should consider providing this out of the box for people to drop files into...
Comment 4 David Liddle 2023-03-06 15:38:28 UTC
Thank you! That is a fantastic suggestion. Your idea of making such a function available out of the box is excellent. As soon as I could, I logged into what you might call our development server, created the directory and alias, and then also added the directory to the <access_dir> section in koha-conf.xml.

- I did have to figure out that, when you wrote ${instance}, it wasn't something of a wildcard entry that Apache would apply to any given instance.
- I'm not quick enough with regular expressions to know if one could enter a single alias in such a way that only the web folder of the applicable instance has its files served by the instance's base URL.
- The pickle of this method is that /etc/koha/apache-shared-opac.conf is also not included by koha-dump. So if one uses koha-restore, one still has to document and implement the changes to that file. Unless, I suppose, a web folder is supplied from the start and a clever alias added to the apache-shared conf.

- IF such a thing was built in, I wonder if it would be good or bad to add the web folder to the Upload tool or have a separate page of its own.

- Basically, I would be in favor of enabling superlibrarians to use nothing but the staff interface to add any type of file that would be referenced by URLs that are manually entered the system preferences. I don't know what percentage of superlibrarians also have sufficient access to their server to upload such files.

I know that much of the above would really belong in a new feature bug describing the addition and function of a web folder. Who should take on the creation of that? As for this bug, I think your suggestion renders it pretty squashed.
Comment 5 David Cook 2023-03-06 22:53:30 UTC
(In reply to David Liddle from comment #4)
> Thank you! That is a fantastic suggestion. Your idea of making such a
> function available out of the box is excellent. As soon as I could, I logged
> into what you might call our development server, created the directory and
> alias, and then also added the directory to the <access_dir> section in
> koha-conf.xml.

I haven't used the access_dir feature but I think that might be defeating the purpose of the access_dir feature. Not a big deal if you do it intentionally I suppose but something to keep in mind, especially if anyone takes over your system, as they might think they're putting files in a private directory and not realizing they're being exposed publicly via Apache.

> - I did have to figure out that, when you wrote ${instance}, it wasn't
> something of a wildcard entry that Apache would apply to any given instance.
> - I'm not quick enough with regular expressions to know if one could enter a
> single alias in such a way that only the web folder of the applicable
> instance has its files served by the instance's base URL.

I'm not sure what you mean by this. The ${instance} is a variable used in the Apache configuration. You can see examples of it throughout the Koha Apache configuration like for the sitemap on /etc/koha/apache-shared-opac.conf

It's defined with this directive (where "kohadev" would be the name of your instance): Define instance "kohadev"

> - The pickle of this method is that /etc/koha/apache-shared-opac.conf is
> also not included by koha-dump. So if one uses koha-restore, one still has
> to document and implement the changes to that file. Unless, I suppose, a web
> folder is supplied from the start and a clever alias added to the
> apache-shared conf.

It is a bit of a pickle. 

> - IF such a thing was built in, I wonder if it would be good or bad to add
> the web folder to the Upload tool or have a separate page of its own.

If you're happy for the files to be managed via the Web UI, then you can already use the "Upload" tool. You just need to tick "Allow public downloads". 

> I know that much of the above would really belong in a new feature bug
> describing the addition and function of a web folder. Who should take on the
> creation of that? As for this bug, I think your suggestion renders it pretty
> squashed.

You've read my mind ;).

I'll mark this one as closed, and you can open a new one describing the behaviour that you're envisioning. Although I think what you're describing is really already possible using the existing "Upload" tool. Personally, I probably wouldn't work on a web tool for this.

I already use something like "/var/lib/koha/${instance}/web" although I maintain that configuration using a customized Koha Debian package. I'd be interested in upstreaming that change. I think I'll open a report for that now...
Comment 6 David Cook 2023-03-06 22:55:03 UTC
Btw one downside of the "Upload" tool is that it serves resources using the Starman server rather than directly via the Apache. 

Apache httpd is more efficient and tuneable for serving static assets, so it tends to be a better choice...
Comment 7 David Liddle 2023-03-07 12:45:43 UTC
(In reply to David Cook from comment #5)
> I'm not sure what you mean by this. The ${instance} is a variable used in
> the Apache configuration. You can see examples of it throughout the Koha
> Apache configuration like for the sitemap on
> /etc/koha/apache-shared-opac.conf
> 
> It's defined with this directive (where "kohadev" would be the name of your
> instance): Define instance "kohadev"

Indeed, it does work; the error was mine. I realized today that, with Plack enabled, I needed to add a few lines to /etc/koha/apache-shared-opac-plack.conf. All is well.
Comment 8 David Cook 2023-03-07 22:22:24 UTC
(In reply to David Liddle from comment #7)
> Indeed, it does work; the error was mine. I realized today that, with Plack
> enabled, I needed to add a few lines to
> /etc/koha/apache-shared-opac-plack.conf. All is well.

You shouldn't need to add a few lines to /etc/koha/apache-shared-opac-plack.conf if you've already added them to /etc/koha/apache-shared-opac.conf