| Summary: | Migrate some SIP configuration into the staff interface | ||
|---|---|---|---|
| Product: | Koha | Reporter: | Martin Renvoize (ashimema) <martin.renvoize> |
| Component: | System Administration | Assignee: | Pedro Amorim <pedro.amorim> |
| Status: | Passed QA --- | QA Contact: | Kyle M Hall (khall) <kyle> |
| Severity: | new feature | ||
| Priority: | P5 - low | CC: | andrew.auld, andrew, blawlor, caroline.cyr-la-rose, carthur, cbrannon, dcook, george, gmcharlt, jonathan.druart, jonathan.field, jvinke, kyle, lisette, lmstrand, lucas, m.de.rooy, magnus, marion.durand, martin.renvoize, nick, pedro.amorim, philippe.blouin, rcoert, wizzyrea |
| Version: | Main | Keywords: | rel_25_11_candidate, RM_priority |
| Hardware: | All | ||
| OS: | All | ||
| URL: | https://github.com/openfifth/koha/commits/final_bug_37893 | ||
| See Also: |
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35369 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41215 |
||
| GIT URL: | Initiative type: | --- | |
| Sponsorship status: | --- | Comma delimited list of Sponsors: | |
| Crowdfunding goal: | 0 | Patch complexity: | Large patch |
| Documentation contact: | Documentation submission: | ||
| Text to go in the release notes: | Version(s) released in: | ||
| Circulation function: | |||
| Bug Depends on: | 38201, 39091, 41167 | ||
| Bug Blocks: | 37248, 40120, 41178, 41179, 41180, 41181, 41214 | ||
| Attachments: |
SIPconfig.xml into DB schema
SIPconfig.xml into DB schema Page issues showing in console |
||
|
Description
Martin Renvoize (ashimema)
2024-09-11 14:27:43 UTC
Created attachment 172530 [details]
SIPconfig.xml into DB schema
Hi all I've started looking into this.
@Kyle @Nick, can I ask a few minutes of your time to briefly look at this schema before I fully commit to it. I aimed to catch and accommodate all possible configurations values from SIPconfig.xml. Thank you.
Created attachment 172584 [details]
SIPconfig.xml into DB schema
sip_institution should be a 1 to 1 with sip_policy, even though SIPconfig.xml technically allows for multiple policies per institution, the template SIPconfig.xml only has one policy per institution and as I understand it the SIP code logic expects only one policy per institution.
This being the case, and policies not relating to anything else, there's no reason to have a separate table for policies specifically and these columns can be merged into sip_institutions instead. Thoughts?
(In reply to Pedro Amorim from comment #2) > Created attachment 172584 [details] > SIPconfig.xml into DB schema > > sip_institution should be a 1 to 1 with sip_policy, even though > SIPconfig.xml technically allows for multiple policies per institution, the > template SIPconfig.xml only has one policy per institution and as I > understand it the SIP code logic expects only one policy per institution. > This being the case, and policies not relating to anything else, there's no > reason to have a separate table for policies specifically and these columns > can be merged into sip_institutions instead. Thoughts? I agree with everything you wrote. I do note the image has institutions and accounts being one to one, but they should be one to many. (In reply to Kyle M Hall (khall) from comment #3) > I do note the image has institutions and > accounts being one to one, but they should be one to many. 100%. Tyvm. Hi all, this is ready for testing.
I'm pasting here the same test plan that is present on the last commit of the branch, the migration test commit:
Test plan for the whole module:
1) Checkout the branch and reset_all
2) Visit the administration page:
/cgi-bin/koha/admin/admin-home.pl
3) Notice the new 'Self-service circulation (SIP2)' item under 'Patrons
and circulation'. Click it.
4) Notice you are now on the new SIP2 module. Create some institutions,
accounts, listeners, etc. Play around with the forms.
5) Notice everytime you add/edit a new SIP2 'resource', a prompt asking
to restart the SIP2 server is shown. If you confirm it, it will
restart the SIP server in order for the new data to take effect.
6) Run 'updatedatabase'. Notice the contents from
/etc/koha/sites/kohadev/SIPConfig.xml have been migrated to the
database are should be accessible in the UI.
7) Confirm all the data has been migrated correctly, and is editable in
the UI as expected.
8) You can repeat this test plan but edit the contents of
/etc/koha/sites/kohadev/SIPConfig.xml with a more complex and
thorough configuration (something resembling a real SIPConfig.xml)
9) After updating the contents of SIPConfig.xml, restart SIP:
koha-sip --restart
10) Test the module again, verify the contents have been migrated as
expected.
11) Run the migration test again to ensure that SIP2 is being served the
exact same configuration before and after the migration:
prove t/db_dependent/Koha/SIP2/SIP2ModuleMigration.t
I'm getting error: DEV atomic update /home/lmstrand/Koha25/installer/data/mysql/atomicupdate/bug_37893.pl [14:58:31]: Bug 37893 - Move SIPconfig.xml to database ERROR: {UNKNOWN}: DBI Exception: DBD::mysql::db do failed: Can't create table `kohacomm`.`sip_accounts` (errno: 150 "Foreign key constraint is incorrectly formed") at /home/lmstrand/Koha25/C4/Installer.pm line 771 Seems the table sip_institutions needs to be generated first. We also have multiple sip config files on our server (under SIPconfig path) that should be iterated through and the file locations are on a different location where the atomicupdate is trying to find them. Maybe the path to the SIPConfig files could be set and read from koha-conf.xml or something? Maybe the passwords could be obscured by ***** by default in the sip2 accounts view and could be shown only when a "view password" button is clicked or something for security reasons? "We also have multiple sip config files on our server (under SIPconfig path) that should be iterated through" This could be difficult to do and managed via the UI, so as a workaround combining our config files into one and running just that sip server process would be an ok solution for us. Anyways I'm thrilled about this, good work! Hi Lari, your feedback and testing is very much appreciated. Thank you for looking. (In reply to Lari Strand from comment #6) > I'm getting error: DEV atomic update > /home/lmstrand/Koha25/installer/data/mysql/atomicupdate/bug_37893.pl > [14:58:31]: Bug 37893 - Move SIPconfig.xml to database > ERROR: {UNKNOWN}: DBI Exception: DBD::mysql::db do failed: Can't create > table `kohacomm`.`sip_accounts` (errno: 150 "Foreign key constraint is > incorrectly formed") at /home/lmstrand/Koha25/C4/Installer.pm line 771 This was an oversight, I've corrected it (sip_institutions needs to be created before sip_accounts) and squashed to https://github.com/PTFS-Europe/koha/commit/c0937ae0b8644fd42c4b84687235407cfb29e73a (In reply to Lari Strand from comment #7) > We also have multiple sip config files on our server (under SIPconfig path) > that should be iterated through and the file locations are on a different > location where the atomicupdate is trying to find them. Maybe the path to > the SIPConfig files could be set and read from koha-conf.xml or something? This is an interesting use case, apologies for my ignorance but how exactly do you have this set up? i.e. how does Koha natively know that it needs to look for multiple files on a certain directory? (In reply to Lari Strand from comment #8) > Maybe the passwords could be obscured by ***** by default in the sip2 > accounts view and could be shown only when a "view password" button is > clicked or something for security reasons? I love this idea, this needs to be built into bug bug 38201 as a "password" type form field, on the framework, and then all modules would be able to use it, including this work here of course. (In reply to Lari Strand from comment #9) > "We also have multiple sip config files on our server (under SIPconfig path) > that should be iterated through" This could be difficult to do and managed > via the UI, so as a workaround combining our config files into one and > running just that sip server process would be an ok solution for us. Coming back to this, is your setup a custom thing or a Koha native thing? If Koha natively supports multiple SIPconfig.xml files then this work needs to consider that. I've not considered it because I'm not aware of it being possible but I'm not very knowledgeable in how SIP may or may not be configured. If this is something you have set up in a custom manner, then yes I think adapting your custom set up for when upgrading into this module is the best course of action. We've been testing this with much success! We've successfully configured connections and run tests of most available options. We have a few things yet to confirm. Right up front, though, we're having trouble with the modal to restart SIP after editing the configuration. SIP did not restart when we selected Yes. Our changes did not save when we selected No. Either way, the modal kept coming back even though we checked the "Don't ask again" checkbox. Less pressingly, it'd be nice to get tooltips in the Institutions setup explaining what's wanted in each field. Thank you so much Andrew for your reply and your time. I've taken note of this and there is indeed a problem in the SIP restart logic (it's not working). I've been meaning to come back to this but it's not been possible, yet! I'm aiming to follow-up here by end of this week or during hackfest next week. Thank you. Hi all, hi Andrew. Thank you for your patience here. I've updated the branch to pick up on database changes without having to restart the SIP server. The latest commit as an additional test plan to test this specific functionality: Changes are picked up by SIP without restart: Test plan: 1) Using KTD, after having followed the original test plan, run the following SIP emulator command: $ perl misc/sip_cli_emulator.pl -su koha -sp koha -l kohalibrary --host localhost --patron 42 -m patron_information 2) Note that the response message contains 'Greetings from Koha'. 3) Using the module UI, visit SIP accounts: http://localhost:8081/cgi-bin/koha/sip2/accounts 4) Find the SIP account that has 'koha' as login. Click edit. 5) Scroll down to 'SIP response mappings' and expand 'Add new screen msg regex'. 6) On 'Find', enter 'Greetings from Koha'. On 'Replace', enter 'Hello from 37893 test plan'. Save. 7) Repeat the sip command from 1). Note that 'Greetings from Koha has been replaced with 'Hello from 37893 test plan', without having to restart the SIP server. Failed QA At step 4 it is unclear what value should be set in implementation field. At step 6, nothing will import if you have created institutions, accounts, etc. Can we please have a more detailed test plan that covers all of the patches? We think this is a really nice feature though! I could see adding "accounts" and "institutions", but not "listeners". I suppose for a Koha system where they don't have sysadmin it would be handy if they could define "listeners", but for a multi-tenanted system you don't want Koha instances being able to define the listeners. Test plan for the whole module: [MODULE UI TESTING] 1) Checkout the branch and reset_all 2) Run 'updatedatabase'. The contents from /etc/koha/sites/kohadev/SIPConfig.xml have been migrated to the database and should be accessible in the UI. 3) Visit the administration page: http://localhost:8081/cgi-bin/koha/admin/admin-home.pl 4) Notice the new 'Self-service circulation (SIP2)' item under 'Patrons and circulation'. Click it. 5) Notice you are now on the new SIP2 module. Create some institutions, accounts, listeners, etc. Play around with the forms. 6) Confirm all the data has been migrated correctly, and is editable in the UI as expected. [SIP TESTING] 1) On ktd, run the following SIP emulator command: $ perl misc/sip_cli_emulator.pl -su koha -sp koha -l kohalibrary --host localhost --patron 42 -m patron_information 2) Note that the response message contains 'Greetings from Koha'. 3) Using the module UI, visit SIP accounts: http://localhost:8081/cgi-bin/koha/sip2/accounts 4) Find the SIP account that has 'koha' as login. Click edit. 5) Scroll down to 'SIP response mappings' and click 'Add new screen msg regex'. 6) On 'Find', enter 'Greetings from Koha'. On 'Replace', enter 'Hello from 37893 test plan'. Save. 7) Repeat the sip command from 1). Note that 'Greetings from Koha has been replaced with 'Hello from 37893 test plan', without having to restart the SIP server. [OPTIONAL] 1) You can repeat the previous steps in [MODULE UI TESTING] but edit the contents of /etc/koha/sites/kohadev/SIPConfig.xml beforehand with a more complex and thorough configuration (something resembling a real SIPConfig.xml) 2) After updating the contents of SIPConfig.xml, restart SIP: koha-sip --restart 3) Test the module again, verify the contents have been migrated as expected. (In reply to John Vinke from comment #14) > Failed QA Hi John, thank you. > At step 4 it is unclear what value should be set in implementation field. I pushed a new commit that should address this: https://github.com/openfifth/koha/commit/871ed139317ea56799909b5af8190b15470a02ee Please let me know. > At step 6, nothing will import if you have created institutions, accounts, > etc. > > Can we please have a more detailed test plan that covers all of the patches? I agree. I've made the previous 2 test plan comments obsolete to avoid causing future confusion. I've updated the test plan on the last tests patch on the branch and also copied here on my previous comment. The revised test plan should no longer cause you to experience this, i.e. first creating manually then import through updatedatabase. However, updatedatabase not adding subsequent data is intentional, as the data being re-inserted will fail due to duplicate keys, but the 'INSERT IGNORE INTO' skips the error. Furthermore, because updatedatabase is intended to run at upgrade time, it is not expected to have database data at that point, only data in the SIPconfig.xml file. If you want to run updatedatabase multiple times to test different setups or SIPconfig.xml files, the data should be removed beforehand to avoid duplicate clashes: delete from sip_accounts; delete from sip_institutions; delete from sip_listeners; delete from sip_server_params; delete from sip_system_preference_overrides; (In reply to David Cook from comment #16) > I could see adding "accounts" and "institutions", but not "listeners". > > I suppose for a Koha system where they don't have sysadmin it would be handy > if they could define "listeners", but for a multi-tenanted system you don't > want Koha instances being able to define the listeners. The idea is to move XML file based configurations into a UI module. I agree that listeners are not quite the same as accounts or institutions, but the alternative would be to end up having half of SIP config in the UI module, and the other half in the XML file. The way this has been implemented is to fully move everything out of the XML file into the new UI module. I'm out of my depth when it comes to the inner workings of SIP, and the following may be a dumb question, but how does one know if a system is multi-tenanted or not? The UI configuration of listeners could be tied to this, i.e. only allow CRUD of listeners if not multi-tenanted. > I'm out of my depth when it comes to the inner workings of SIP, and the following may be a dumb question, but how does one know if a system is multi-tenanted or not? The UI configuration of listeners could be tied to this, i.e. only allow CRUD of listeners if not multi-tenanted.
I think multi-tenanted here means "more than one instance running on the same server". You don't want different instances to set up their SIPservers to listen on the same port.
Not sure how to solve that in code, though. It might have to require people from the different instances to just talk to each other?
(In reply to Magnus Enger from comment #20) > I think multi-tenanted here means "more than one instance running on the > same server". You don't want different instances to set up their SIPservers > to listen on the same port. > > Not sure how to solve that in code, though. It might have to require people > from the different instances to just talk to each other? Multiple Koha instances, got it. The migration process is designed to pick up the correct respective SIPconfig.xml for the instance running the upgrade so it should only import it's respective data (including listeners). In relation to the issue of different listeners (from different Koha instances) using the same port, as far as I can tell this is not checked/enforced currently, i.e. this is already an issue/possibility before moving it into the module, though I agree that it may expose / make this more prone to happening since it'll become editable in the UI. I'm chatting with Martin about a possible solution we could try here. Magnus and David, please take a look at this commit when you have the chance: https://github.com/openfifth/koha/commit/a0a003a5aaec12ac8103c3b4512478ddda1fbb8b (In reply to Pedro Amorim from comment #21) > In relation to the issue of different listeners (from different Koha > instances) using the same port, as far as I can tell this is not > checked/enforced currently, i.e. this is already an issue/possibility before > moving it into the module, though I agree that it may expose / make this > more prone to happening since it'll become editable in the UI. (In reply to Pedro Amorim from comment #22) > Magnus and David, please take a look at this commit when you have the chance: > https://github.com/openfifth/koha/commit/ > a0a003a5aaec12ac8103c3b4512478ddda1fbb8b If you have 1 sysadmin managing N SIPconfig.xml, you have 1 person managing the IP addresses and ports across all the instances though, so conflicts aren't a problem. What I'm trying to say is that - as a vendor sysadmin - I don't want Koha library admins being able to configure the IP address and port number for the listener. There would be some potential for malicious abuse to the underlying host system if a user was able to dictate the IP address and port used. It would probably be somewhat limited since it would be running as the Koha user, so it wouldn't be able to bind to a privileged port, but it's still not great. Realistically, in production, the sysadmin will have likely set up stunnel and have that proxying to specific ports they've defined in SIPconfig.xml. There's also the case of the sysadmin or network admin setting up the firewalls. So I'm not quite sure of the benefit to moving the listener config into the web UI? I can only see the downsides. (In reply to Pedro Amorim from comment #22) > Magnus and David, please take a look at this commit when you have the chance: > https://github.com/openfifth/koha/commit/ > a0a003a5aaec12ac8103c3b4512478ddda1fbb8b Looking at this again, I think you've got a typo for the $addr? You're trying to put the port into that variable? Even with that validation, the user would still be able to define any non-privileged port and any IPv4 address. (Note that the validation would prevent IPv6 addresses.) -- Just to reiterate... I'm not worried about web users being able to successfully bind to an IP address and port. I'm worried about them binding to addresses and ports that they shouldn't. (Of course, a workaround for that could be to have address and port limits in koha-conf.xml with some sensible defaults. Like a limit on ports 6000-6999 although limits on IP address could be trickier because of differing setups.) This worked great. Andrew and I tested. As far as listeners go, we would have no problem if community wanted to not make listeners editable. It seems like the biggest place things could go wrong with this. It would be nice to see the information, but editing it could cause big headaches. (In reply to Lisette Scheer from comment #25) > This worked great. Andrew and I tested. > As far as listeners go, we would have no problem if community wanted to not > make listeners editable. It seems like the biggest place things could go > wrong with this. It would be nice to see the information, but editing it > could cause big headaches. That's a good point about being able to see the information. It could be useful to let library admins see the information, so that they can have more informed discussions with their IT. *** Bug 33681 has been marked as a duplicate of this bug. *** Tested this successfully and I'm signing off on it. I'm still not keen on this idea, but looking at the new_bug_37893 branch at a glance... it looks like you can still use the existing SIPconfig.xml and I could just locally comment out all the Koha::SIP2::* sections in C4::SIP::Sip::Configuration... I think I am going to mark as Failed QA though, because there's no sanity checking on the Koha::SIP2::ServerParam object. Theoretically, you could DOS a server (accidentally or maliciously) by feeding in server-params that fill up the process table. I agree with the original idea that "accounts" and "institutions" should be moved into the DB/web UI, but "listeners" and "server-params" really should not be. "syspref_overrides"... I'm not familiar enough with the use of that in the SIP server, so I don't have ready comment on that one. (In reply to David Cook from comment #29) > Theoretically, you could DOS > a server (accidentally or maliciously) by feeding in server-params that fill > up the process table. Can you please provide an example / test plan to reproduce / replicate this so I can address it? (In reply to David Cook from comment #30) > I agree with the original idea that "accounts" and "institutions" should be > moved into the DB/web UI, but "listeners" and "server-params" really should > not be. > > "syspref_overrides"... I'm not familiar enough with the use of that in the > SIP server, so I don't have ready comment on that one. You are proposing that only the accounts and institutions part is moved out of SIPConfig.xml. Everything else related to SIP configuration is kept in SIPConfig.xml. Is that correct? @David Cook, would a new 'sip2_module' staff permission here help / be enough? I think that syspref_override car also be migrated in the interface Sorry my bug mail is broken so I didn't get this. Just noticed your response by chance... (In reply to Pedro Amorim from comment #31) > (In reply to David Cook from comment #29) > > Theoretically, you could DOS > > a server (accidentally or maliciously) by feeding in server-params that fill > > up the process table. > > Can you please provide an example / test plan to reproduce / replicate this > so I can address it? Just looking up the process limits now. It looks like filling up the process table might not be possible when running the SIP server as a non-root user, since non-root users should have limits on the number of processes they can run. So it looks like I might be wrong there. But you could still create problems even as a non-root user. Basically just put the your min_servers, max_servers, and min_spare_servers to very high values. For instance, max_servers 500000 and min_spare_servers 450000. Keep in mind that the server-params are passed to the SIP server completely un-checked and un-sanitized, and that all parameters listened in the "CONFIGURATION FILE" section of Net::Server::PreFork can be used. It's just way too much control. > (In reply to David Cook from comment #30) > > I agree with the original idea that "accounts" and "institutions" should be > > moved into the DB/web UI, but "listeners" and "server-params" really should > > not be. > > > > "syspref_overrides"... I'm not familiar enough with the use of that in the > > SIP server, so I don't have ready comment on that one. > > You are proposing that only the accounts and institutions part is moved out > of SIPConfig.xml. Everything else related to SIP configuration is kept in > SIPConfig.xml. Is that correct? More or less. Basically, I'm saying "listeners" and "server-params" should stay in SIPConfig.xml. (In reply to Pedro Amorim from comment #32) > @David Cook, would a new 'sip2_module' staff permission here help / be > enough? Nope, because these are server-level parameters and not Koha/application-level parameters. The only web user permission that could potentially make sense for this is if we created a new user category like "system_administrator" and not give anyone that category except the system administrator that actually manages that server. These are not things that libraries should be able to manage themselves. Just the person managing the server. (In reply to David Cook from comment #34) > table might not be possible when running the SIP server as a non-root user, > since non-root users should have limits on the number of processes they can > run. So it looks like I might be wrong there. > > But you could still create problems even as a non-root user. > > Basically just put the your min_servers, max_servers, and min_spare_servers > to very high values. For instance, max_servers 500000 and min_spare_servers > 450000. If you wanted to check for the DOS, I'd say up those more to 5000000 for max_servers and 4500000 for min_spare_servers. You might need "min_servers" to match "min_spare_servers". I can't quite recall. (Taking a quick look at the code and yeah it looks like min_serves must be more than min_spare_servers at least.) Overall, just try to break stuff. Set log_level to 4 and change log_file to /tmp/hack_file.log or something else writeable by the Koha user (like /etc/koha/sites/<koha_instance>/hack_file.log) and try to fill up the file system and cause crashes that way. (Apologies if it takes me some time to respond. My mail server is blocking Bugzilla so I'm not getting email notifications at this time.) *** Bug 21935 has been marked as a duplicate of this bug. *** Just an idea here (but it might be managed in a separate bug after this one is pushed), it would be amazing to be able to duplicate all translation (screen_msg_regx) from an account to another. And even more to export and import all translation for transfer between Koha. It is currently quite difficult to manage translation when maintaining multiple instances which all have multiple login. (In reply to Marion Durand from comment #39) > Just an idea here (but it might be managed in a separate bug after this one > is pushed), it would be amazing to be able to duplicate all translation > (screen_msg_regx) from an account to another. And even more to export and > import all translation for transfer between Koha. > > It is currently quite difficult to manage translation when maintaining > multiple instances which all have multiple login. Would a 'duplicate' (full resource) feature address this? The idea being you'd duplicate an existing SIP2 account (create a new one with values from previous already filled in), keep the things you want (e.g. screen_msg_regx) and remove/change the rest? I ask this because bug 38201 (that blocks this one) aims to move development towards a more framework oriented approach. If this 'duplicate' feature is developed at the framework level, every VueJS module would be able to make use of it (including this SIP2 module), as this 'duplicate' functionality would certainly be useful in other modules / use cases (?) (In reply to Pedro Amorim from comment #40) > (In reply to Marion Durand from comment #39) > > Just an idea here (but it might be managed in a separate bug after this one > > is pushed), it would be amazing to be able to duplicate all translation > > (screen_msg_regx) from an account to another. And even more to export and > > import all translation for transfer between Koha. > > > > It is currently quite difficult to manage translation when maintaining > > multiple instances which all have multiple login. > > Would a 'duplicate' (full resource) feature address this? The idea being > you'd duplicate an existing SIP2 account (create a new one with values from > previous already filled in), keep the things you want (e.g. screen_msg_regx) > and remove/change the rest? Yes it would definitely be a great help ! Even if a way to export+import screen messages would still be nice as it allows sharing translation across different libraries and instances of Koha. Hi all, I've revisited this: - A third rewrite has been done on the module to adhere to the patterns introduced by pushed bug 38201 (VueJS composition API, etc) - Server params and listeners have been removed from the UI module and are kept in SIPConfig.xml as per QA comments by David Cook. - Cache is now utilized to check for the last time an account, an institution or a system preference override has been updated. When a new SIP request comes in, it checks if SIP configuration has been updated and does a new read from the database if so. This allows for changes that are made by the UI to reflect on the SIP server without needing a server restart. QA script complaints: - Files were added+removed in the same patch. This refers to the server params+listeners functionality that once existed but has since been purged. - The above happens because I've opted to not squash any patches. This project has gone for over a year (on and off) and contains a patch co-authored by Martin Renvoize and a patch co-authored by Mark Hofstetter. - Some database columns are tinyint but not boolean: lost_block_checkout, lost_block_checkout_value, lost_status_for_missing. These are false positives as these are all intentionally tinyint not boolean. - I believe errors on SIPServer.pm related to pod_coverage are false positives coming from old methods that have non-conformant pod code, not by code added on this patchset. Test plan remains the same. However, because many code changes have happened, I'm asking those who tested before to please test again when convenient. Test plan for the whole module: [MODULE UI TESTING] 1) Checkout the branch and reset_all 2) Run 'updatedatabase'. The contents from /etc/koha/sites/kohadev/SIPConfig.xml have been migrated to the database and should be accessible in the UI. 3) Visit the administration page: http://localhost:8081/cgi-bin/koha/admin/admin-home.pl 4) Notice the new 'Self-service circulation (SIP2)' item under 'Patrons and circulation'. Click it. 5) Notice you are now on the new SIP2 module. Create some institutions, accounts, listeners, etc. Play around with the forms. 6) Confirm all the data has been migrated correctly, and is editable in the UI as expected. [SIP TESTING] 1) On ktd, run the following SIP emulator command: $ perl misc/sip_cli_emulator.pl -su koha -sp koha -l kohalibrary --host localhost --patron 42 -m patron_information 2) Note that the response message contains 'Greetings from Koha'. 3) Using the module UI, visit SIP accounts: http://localhost:8081/cgi-bin/koha/sip2/accounts 4) Find the SIP account that has 'koha' as login. Click edit. 5) Scroll down to 'SIP response mappings' and click 'Add new screen msg regex'. 6) On 'Find', enter 'Greetings from Koha'. On 'Replace', enter 'Hello from 37893 test plan'. Save. 7) Repeat the sip command from 1). Note that 'Greetings from Koha has been replaced with 'Hello from 37893 test plan', without having to restart the SIP server. [OPTIONAL] 1) You can repeat the previous steps in [MODULE UI TESTING] but edit the contents of /etc/koha/sites/kohadev/SIPConfig.xml beforehand with a more complex and thorough configuration (something resembling a real SIPConfig.xml) 2) After updating the contents of SIPConfig.xml, restart SIP: koha-sip --restart 3) Test the module again, verify the contents have been migrated as expected. All values from the SIP config imported successfully. All functionality worked as well with these patches as it does in current main. We could not get the default example bin sorting rule to work with this patch, but we couldn't get it to work on current main, either. However, failing QA due to test failures in the QA tool. Test Summary Report ------------------- /kohadevbox/koha/t/db_dependent/api/v1/sip2_accounts.t (Wstat: 512 (exited 2) Tests: 6 Failed: 2) Failed tests: 3, 6 Non-zero exit status: 2 Files=1, Tests=6, 5 wallclock secs ( 0.01 usr 0.00 sys + 3.73 cusr 0.34 csys = 4.08 CPU) Result: FAIL (In reply to Andrew Fuerste-Henry from comment #43) > All values from the SIP config imported successfully. > All functionality worked as well with these patches as it does in current > main. We could not get the default example bin sorting rule to work with > this patch, but we couldn't get it to work on current main, either. > > However, failing QA due to test failures in the QA tool. > > Test Summary Report > ------------------- > /kohadevbox/koha/t/db_dependent/api/v1/sip2_accounts.t (Wstat: 512 (exited > 2) Tests: 6 Failed: 2) > Failed tests: 3, 6 > Non-zero exit status: 2 > Files=1, Tests=6, 5 wallclock secs ( 0.01 usr 0.00 sys + 3.73 cusr 0.34 > csys = 4.08 CPU) > Result: FAIL I can't reproduce this, the QA script only comes back with the false positives I mentioned above, nothing about tests files: Running tests (4) * Proving /kohadevbox/koha/t/db_dependent/Koha/SIP2/SIP2ModuleMigration.t PASS * Proving /kohadevbox/koha/t/db_dependent/api/v1/sip2_accounts.t PASS * Proving /kohadevbox/koha/t/db_dependent/api/v1/sip2_institutions.t PASS * Proving /kohadevbox/koha/t/db_dependent/api/v1/sip2_system_preference_overrides.t PASS Can you please check again? With fresh ktd data only. The test failures I was getting cleared up when testing on fresh KTD data. Signing off! TY! A few tiny QA follow-ups were added to the branch, and signoff and sponsorship lines were added. All in all, I think this is a great piece of work and really moves Koha forward. I have a suggestion for a follow-up which Pedro is aware of for the future, but it more relates to Vue framework than the SIP interfaces here.. my final follow-up notes it and allows for us to move forward without it by adding a trivial tooltip. Passing QA, please grab the branch in your leisure Lucas :) Finally getting around to this... Not planning to be a blocker. Just going to share some notes/observations. If there's anything that needs changing, I figure it can be done in a future follow-up bug. -- Notes: 1. Curious why etc/SIPconfig.xml and debian/templates/SIPconfig.xml have comments added rather than having their legacy sections removed? 2. I would've expected to find "Self-service circulation (SIP2)" under "Additional parameters" rather than "Patrons and circulation" in Admin, but I suppose it makes a certain conceptual sense. 3. The description in on admin-home.pl still includes "server parameters" which I think is incorrect. 4. Curious why "sip2.pl" went into its own "sip2" directory. Guessing it's because of imitating "erm". I don't think this is a good move for the long-term organisation of Koha source code, but c'est la vie I guess. 5. It looks like there is no way to define a password for a SIP2 account in the web ui, but... that's because it uses the actual borrower account, so it's no longer required to store the password in plain text. Oh that's very cool. I can imagine that would confuse some people, but still... very cool. (I also like how you can update the password for a user and you don't have to restart the SIP server. Excellent.) 6. The console is full of "Incorrect use of <label for=FORM_ELEMENT>" page errors. The browser says this could negatively affect accessibility tools. 7. There is no validation on the "Staff userid for SIP2 authentication". You can put anything you like. That one would be cool to be done by a dropdown menu which only shows users that have appropriate permissions. 8. I can't find an easy way to test institutions with the time and data I have at hand right now, so I'm going to just assume they work correctly. It would've been good to have a more comprehensive test plan with things like checkins and checkouts to prove that the change works... Actually I just realised the wiki has some things for this... Hmm... I can't seem to perform SIP checkouts... perl misc/sip_cli_emulator.pl -su koha -sp 1Test1Test -l CPL --host localhost --patron 42 -m checkout --item 39999000001310 Attempting socket connection to localhost:6001...connected! SEND: 9300CNkoha|CO1Test1Test|CPCPL| READ: 941 Trying 'checkout' SEND: 11YN20251030 05164020251030 051640AOCPL|AA42|AB39999000001310|AC1Test1Test|BIN| READ: undef Seeing the following in sip-output.log: [2025/10/30 05:17:48] [23774] [WARN] koha@127.0.0.1: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'branchcode' at row 1 at /kohadevbox/koha/Koha/Object.pm line 174 Try: :Tiny::try /usr/share/perl5/Try/Tiny.pm (123) [2025/10/30 05:17:48] [23774] [ERROR] koha@127.0.0.1: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'branchcode' at row 1 at /kohadevbox/koha/Koha/Object.pm line 174 C4: :SIP::Trapper::__ANON__ /kohadevbox/koha/C4/SIP/Trapper.pm (70) [2025/10/30 05:17:48] [23774] [ERROR] koha@127.0.0.1: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Data too long for column 'branchcode' at row 1 at /kohadevbox/koha/Koha/Object.pm line 174 C4: :SIP::Trapper::__ANON__ /kohadevbox/koha/C4/SIP/Trapper.pm (70) [2025/10/30 05:17:48] [23774] [INFO] koha@127.0.0.1: raw_transport: shutting down C4::SIP::Sip::siplog /kohadevbox/koha/C4/SIP/Sip.pm (327) Ah that's because I'd updated the account to use "kohalibrary" in Accounts which is an invalid institution. Changed it back to CPL and now it's working. 9. Hmmm I turned off "Checkin" for the CPL institution but it still let me checkin... maybe it does require a restart. I'll re-test that... Hmm nope... still allows checkin after a restart of the SIP server. Although to be fair I don't know if any of those attributes ever worked in SIPconfig.xml either. -- Overall, it's looking pretty good I think. I think it's a good move in the right direction for sure. Just reiterating that I'm giving this a +1 from me. I think if any issues do pop up... we fix them as they come. This is a good generational shift for Koha's SIP2 server I think. (In reply to David Cook from comment #47) > Finally getting around to this... > > Not planning to be a blocker. Just going to share some notes/observations. > If there's anything that needs changing, I figure it can be done in a future > follow-up bug. Thank you for your time here David. > > Notes: > 1. Curious why etc/SIPconfig.xml and debian/templates/SIPconfig.xml have > comments added rather than having their legacy sections removed? Fair, I'm not sure exactly why I decided this way, but I agree that it should be remove, although as you say trivial and non-blocker. > 2. I would've expected to find "Self-service circulation (SIP2)" under > "Additional parameters" rather than "Patrons and circulation" in Admin, but > I suppose it makes a certain conceptual sense. Up for debate I suppose, personally don't feel strongly about this and happy to follow-up if required. > 3. The description in on admin-home.pl still includes "server parameters" > which I think is incorrect. Good spot. Patch provided: https://github.com/openfifth/koha/commit/fa5d3b678123e1f8d5d968923897861006486e2e > 4. Curious why "sip2.pl" went into its own "sip2" directory. Guessing it's > because of imitating "erm". I don't think this is a good move for the > long-term organisation of Koha source code, but c'est la vie I guess. Correct, to adhere to established pattern. > 5. It looks like there is no way to define a password for a SIP2 account in > the web ui, but... that's because it uses the actual borrower account, so > it's no longer required to store the password in plain text. Oh that's very > cool. I can imagine that would confuse some people, but still... very cool. > (I also like how you can update the password for a user and you don't have > to restart the SIP server. Excellent.) > > 6. The console is full of "Incorrect use of <label for=FORM_ELEMENT>" page > errors. The browser says this could negatively affect accessibility tools. I cannot see this. Can you please provide more information e.g. which page, what browser. > 7. There is no validation on the "Staff userid for SIP2 authentication". You > can put anything you like. That one would be cool to be done by a dropdown > menu which only shows users that have appropriate permissions. This is the follow-up suggestion Martin is referring to in his comment. We discussed and agreed that a 'patron search' relationship type of input is required to be added to the framework for this, but a wider discussion with the community is needed to agree on a way of doing (use patron_autocomplete? Use existing patron search modal in vuejs?) > > 9. Hmmm I turned off "Checkin" for the CPL institution but it still let me > checkin... maybe it does require a restart. I'll re-test that... > > Hmm nope... still allows checkin after a restart of the SIP server. > > Although to be fair I don't know if any of those attributes ever worked in > SIPconfig.xml either. I've looked here and these seem to not be working, even before this piece. It doesn't matter if checkin/checkout if 'true', 'false', or non-existent, it always allows for checkin/checkout. I thought it could be because I was testing checkin/checkout for cardnumber '42' (superlibrarian) but using a regular patron produces the same results. I'd consider this to be a bug outside the scope of this work. That all sounds good, Pedro. Thanks for reviewing my notes. (In reply to Pedro Amorim from comment #49) > > 6. The console is full of "Incorrect use of <label for=FORM_ELEMENT>" page > > errors. The browser says this could negatively affect accessibility tools. > > I cannot see this. Can you please provide more information e.g. which page, > what browser. Yep, my bad. I should've been more specific. I can't recall exactly now, but I do have one example which I'll explain and provide a screenshot for. Steps: 0. Open F12 tools (I'm in Chrome but could be Firefox, Edge, whatever) and go to "Console" tab 1. Go to http://localhost:8081/cgi-bin/koha/sip2/sip2.pl 2. Click "Accounts" 3. Click "Edit" on any of the rows 4. In the upper right hand corner, there should be a little red flag with an X on it and a count of around 28 next to it. These are issues that don't generate console messages but still show up as issues in the browser. I think we often ignore them in Koha (and most web apps tbh), but I thought I'd flag it anyway. Created attachment 188721 [details]
Page issues showing in console
(In reply to David Cook from comment #50) > > I cannot see this. Can you please provide more information e.g. which page, > > what browser. > > Yep, my bad. I should've been more specific. I can't recall exactly now, but > I do have one example which I'll explain and provide a screenshot for. Looking at this again and I think I see what it means. While many labels have a "for" attribute, they're not for a single form input, so they're not actually usable. "Payment type writeoff" works because it's a simple mapping between the label and the input, but then things like "Holds block checkin" or "Cash register" show up as violations. (In reply to David Cook from comment #52) > (In reply to David Cook from comment #50) > > > I cannot see this. Can you please provide more information e.g. which page, > > > what browser. > > > > Yep, my bad. I should've been more specific. I can't recall exactly now, but > > I do have one example which I'll explain and provide a screenshot for. > > Looking at this again and I think I see what it means. While many labels > have a "for" attribute, they're not for a single form input, so they're not > actually usable. I see it now as well, yes. This appears to be an issue in the framework itself, if you create a new ERM agreement you get the same errors: http://localhost:8081/cgi-bin/koha/erm/agreements/add I think that's out of scope here, v-select elements seem to trigger this warning, boolean/radio elements also seem to trigger this warning due to the way HTML elements are rendered. For the institutions editor: It appears that deleting an institution will delete all accounts connected to that institution ( which makes sense ) with no warning. It would be best if the editor either warned that "the following accounts will be deleted: act1, act2, etc." or simply prevent deletion of institutions with SIP accounts. It would also be nice if the institution "name" field were limited to a branchcode pulldown since Koha can no longer handle institutions that are not branchcodes. This is a bit out of scope so it's not a blocker. At the very least the tooltip should state that unexpected behavior can occur if it is *not* a valid branchcode. The policy defaults should all default to "yes" in my mind, but I'm open to arguments against it. Retries field should be limited to a max of 999 retries, as the field is 3 characters long. This is a blocker and a minor fix I think. "Staff userid for SIP2 authentication" could be using the patron search in the same way that circulation does to fill in the userid from a list of options. Also not a blocker but would be really nice to have. For the accounts editor: "Convert nonprinting characters" defaults to NULL for imported accounts, but defaults to empty string for created accounts with no option for NULL. The only value I've ever used for this is a space, as it is usually used to convert things like newlines for devices that cannot handle them. This could be either changed to a text field with a checkbox, or a pulldown with options for NULL and space ( and could be added to in the future ), or even a list of authorized values. The simplest solution would be to have empty string convert to NULL. Any of these solutions is acceptable IMO. "Format due date" should have a tooltip explaining it will format the due date in the current DateFormat syspref. "Holds get captured" may need a tool tip? Not a blocker, but I'm not sure how well understood this one will be. "Previous checkout block checkout" should read "Block checkout of items previously or currently checked out by a patron", or maybe "Block previous checkouts" with a tooltip since that is quite verbose. "Send patron home library in AF" can be deprecated but is totally out of scope for this bug. It was for Overdrive but they no longer use this feature ( though we'll need to verify they don't have legacy customers using it if we do ). They now use custom fields of the patron's branchcode. Terminator should default to CRLF. Patron attribute could use a pulldown of attribute types. Could use a tooltip at the least. It would be nice if delimiter, encoding and terminator had the defaults inserted into the database instead of being NULL in the database and showing as empty cells in the accounts table. SIP communication itself appears to work great! (In reply to Kyle M Hall (khall) from comment #54) Thank you so much for your time looking here, Kyle. > For the institutions editor: > It appears that deleting an institution will delete all accounts connected > to that institution ( which makes sense ) with no warning. It would be best > if the editor either warned that "the following accounts will be deleted: > act1, act2, etc." or simply prevent deletion of institutions with SIP > accounts. Considering this non-blocker, I hope you understand and agree, for the following reasons: 1) A confirmation box already exists. 2) Only authorized staff members with assigned permissions are able to delete institutions in the first place. 3) This needs a framework doResourceDelete change (i.e. may need additional testing / cause a QA delay here). For that, I've opened bug 41178, and provided patches there. I don't think full module should be blocked by this, and ideally bug 41178 is also looked at by Matt+Joubu. > It would also be nice if the institution "name" field were limited to a > branchcode pulldown since Koha can no longer handle institutions that are > not branchcodes. This is a bit out of scope so it's not a blocker. At the > very least the tooltip should state that unexpected behavior can occur if it > is *not* a valid branchcode. Not a blocker. Opened bug 41180 for future enhancement. Patch updating the tooltip pushed to branch: https://github.com/openfifth/koha/commit/3ecf278a05ed4dc96e0d104ce0d91ce6626a98ac > The policy defaults should all default to "yes" in my mind, but I'm open to > arguments against it. Patch submitted: https://github.com/openfifth/koha/commit/0278df6b9575846c4b67fbae0619cbbe4dd56d94 > Retries field should be limited to a max of 999 retries, as the field is 3 > characters long. This is a blocker and a minor fix I think. Patch submitted: https://github.com/openfifth/koha/commit/24393d6d0b46a976f358673ebda9af37f8d059dd > "Staff userid for SIP2 authentication" could be using the patron search in > the same way that circulation does to fill in the userid from a list of > options. Also not a blocker but would be really nice to have. Not a blocker. Opened bug 41179 for future enhancement. > For the accounts editor: > "Convert nonprinting characters" defaults to NULL for imported accounts, but > defaults to empty string for created accounts with no option for NULL. The > only value I've ever used for this is a space, as it is usually used to > convert things like newlines for devices that cannot handle them. This could > be either changed to a text field with a checkbox, or a pulldown with > options for NULL and space ( and could be added to in the future ), or even > a list of authorized values. The simplest solution would be to have empty > string convert to NULL. Any of these solutions is acceptable IMO. Patch ensure empty string is submitted as NULL: https://github.com/openfifth/koha/commit/95e718520f2d9a3bff1f2ed12e2d735b925806c7 > "Format due date" should have a tooltip explaining it will format the due > date in the current DateFormat syspref. Patch updating tooltip: https://github.com/openfifth/koha/commit/6d3373f7ce0565c44a7dda9c78b7ec2fd8e87466 > "Holds get captured" may need a tool tip? Not a blocker, but I'm not sure > how well understood this one will be. Patch updating tooltip: https://github.com/openfifth/koha/commit/8a59f8935ca82975f5a24a2d849d157bb352e2f3 > "Previous checkout block checkout" should read "Block checkout of items > previously or currently checked out by a patron", or maybe "Block previous > checkouts" with a tooltip since that is quite verbose. Patch updating label+tooltip: https://github.com/openfifth/koha/commit/019a5d6a511631f793e083ac292b1772fe04296d > "Send patron home library in AF" can be deprecated but is totally out of > scope for this bug. It was for Overdrive but they no longer use this feature > ( though we'll need to verify they don't have legacy customers using it if > we do ). They now use custom fields of the patron's branchcode. No action made here. I'll leave it up to you if you feel necessary to open a new bug for this or not. > Terminator should default to CRLF. Patch defaulting terminator to CRLF in form: https://github.com/openfifth/koha/commit/412003aeb35d3b5096098d342d0544fa9406ad20 > Patron attribute could use a pulldown of attribute types. Could use a > tooltip at the least. Opened bug 41181 for future enhancement. Patch updating field tooltip in form: https://github.com/openfifth/koha/commit/81408e0c1dfd54d74aa49ed68001a3a4f41c2d5a > It would be nice if delimiter, encoding and terminator had the defaults > inserted into the database instead of being NULL in the database and showing > as empty cells in the accounts table. Patch defaulting delimiter and terminator inserted values in database: https://github.com/openfifth/koha/commit/53670bebfd78bf2eac2ee51ae131f38a38483ca3 encoding was skipped as I'm unsure if it should default to utf or ascii and the code appears to consider NULL as a possibility (?). I'm setting this back to PQA as having addressed the feedback but I'm happy to discuss further and/or revisit anything that needs it. Thank you for the followup bugs and patches! I think this is ready to be pushed! There are tons of failures caught by the QA script. Quick QA review (not tested SIP)
1. There are no cypress tests
2. Only use the plural module is needed
eg.
22 use Koha::SIP2::Account;
23 use Koha::SIP2::Accounts;
same pattern in several places
3. Tests are missing for (at least) Koha/SIP2/Account.pm
Also get_for_config sub are not covered by tests. Not sure how it would be useful however...
4. If I upgrade I do have stuffs in the DB (picked from /etc/koha/sites/kohadev/SIPconfig.xml) but for new installs the table are not populated.
Maybe we need to patch miscdev(?)
5. "Cash register:" shouldn't we hide it if none exists or if the pref is off?
6. Framework bugs?
a. you can enter a letter after a number ("42a") in the numeric input (eg. "retries") and submit the form and you get a "400 Something went wrong: Error: Expected integer - got string."
b. you can enter a letter in the number inputs ('x') and it becomes "NaN"
7 Don't we want a config flag to turn off the ability to edit from the UI?
8. kohastructure.sql has wrong DROP/CREATE statement
+DROP TABLE IF EXISTS `file_transports`;
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE `file_transports` (
9. I think sip2/sip2.tt is wrong about #sip2 being closed in intranet-bottom.inc
It should be <div id="sip2"></div> and the last closing div removed.
10. Breadcrumb start with SIP2, shouldn't it be "Administration"?
11. Missing changes in etc/koha-httpd.conf? Nice catches there, Jonathan. (In reply to Jonathan Druart from comment #59) > 11. Missing changes in etc/koha-httpd.conf? I haven't checked all the places Jonathan mentioned but I have checked this one and in light of bug 41167, I'm marking this as Failed QA Thank you Jonathan for the time spent here, as always. I've done my best to address your feedback below. (In reply to Jonathan Druart from comment #57) > There are tons of failures caught by the QA script. Yes, please read comment 42: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37893#c42 I believe most of these are false positives, however the following follow-up patches fix those that aren't: 1) Set tinyint boolean => 0 for tinyint not boolean fields: https://github.com/openfifth/koha/commit/521a53224c4d364e2910397a801a1c4af252b9c5 2) Add new line to apache-shared-intranet.conf: https://github.com/openfifth/koha/commit/ea30ba55dd978c92e33e6f270a8e7931e7d79d69 3) js-patron-format.inc (only shows now after bug 40958): https://github.com/openfifth/koha/commit/4ee64da97a6e7b7d60cec65131fd3ea925da3365 (In reply to Jonathan Druart from comment #58) > Quick QA review (not tested SIP) > > 1. There are no cypress tests Yes, there are. See t/cypress/integration/SIP2. > 2. Only use the plural module is needed > eg. > 22 use Koha::SIP2::Account; > 23 use Koha::SIP2::Accounts; > same pattern in several places Patch addressing: https://github.com/openfifth/koha/commit/c7b1ee47073b6e1e94f92f8cc6a8fa3e4f1ba6b2 > 3. Tests are missing for (at least) Koha/SIP2/Account.pm > Also get_for_config sub are not covered by tests. Not sure how it would be > useful however... Tests added for Koha/SIP2/Account.pm: https://github.com/openfifth/koha/commit/3a07ee8194eb95a2897afc2c3e44a796c7b70975 get_for_config for all SIP related classes is tested by SIP2ModuleMigration.t. This test file tests and proves that the configuration read by the SIPServer is the same before and after the migration from XML to the database, which invokes get_for_config for each of the classes. > 4. If I upgrade I do have stuffs in the DB (picked from > /etc/koha/sites/kohadev/SIPconfig.xml) but for new installs the table are > not populated. > Maybe we need to patch miscdev(?) Out of scope here? AFAIK the 'dummy' existing config in SIPconfig.xml served only as documentation for all the possible configuration values for accounts + institutions + syspref overrides (which is what is migrated here). Server params and listeners are still read from the SIPconfig.xml file (as per DCook's original QA requirements). > 5. "Cash register:" shouldn't we hide it if none exists or if the pref is > off? Patch submitted to bug 41214. I agree with this, however I think a wider 'add option for resource attributes to display based on sys pref' to the VueJS framework is needed here. For this reason, I think this makes more sense as a follow-up bug, to be addressed and discussed in bug 41214. I don't think this should be a blocker here as: 1) The field is optional 2) Information regarding UseCashRegisters sys pref being required is also included in the field's respective tooltip. > 6. Framework bugs? > a. you can enter a letter after a number ("42a") in the numeric input (eg. > "retries") and submit the form and you get a "400 Something went wrong: > Error: Expected integer - got string." > b. you can enter a letter in the number inputs ('x') and it becomes "NaN" Yes, opened bug 41215, with reproduction instructions using existing vendors already in main (i.e. unrelated to the patchset here). > 7 Don't we want a config flag to turn off the ability to edit from the UI? I don't think so? The ability to access and work with the module is already governed by the new 'sip2' permission. > 8. kohastructure.sql has wrong DROP/CREATE statement > > +DROP TABLE IF EXISTS `file_transports`; > +/*!40101 SET @saved_cs_client = @@character_set_client */; > +/*!40101 SET character_set_client = utf8 */; > +CREATE TABLE `file_transports` ( Thank you for spotting this. It must've been caused by a failed past rebase. Fixed here: https://github.com/openfifth/koha/commit/26438204c16d1deaea2757cdd165de0aacbbdb38 > 9. I think sip2/sip2.tt is wrong about #sip2 being closed in > intranet-bottom.inc > It should be <div id="sip2"></div> and the last closing div removed. Patch addressing: https://github.com/openfifth/koha/commit/853f1365103c5ef747d7e136828ce75c21e2bebb > 10. Breadcrumb start with SIP2, shouldn't it be "Administration"? Patch addressing: https://github.com/openfifth/koha/commit/a704e629f5d96b16a284d1e4526de95dcd4c9ce7 (In reply to David Cook from comment #60) > Nice catches there, Jonathan. > > (In reply to Jonathan Druart from comment #59) > > 11. Missing changes in etc/koha-httpd.conf? > > I haven't checked all the places Jonathan mentioned but I have checked this > one and in light of bug 41167, I'm marking this as Failed QA Thank you for bringing my attention to bug 41167, I was not aware of it. However bug 41167 is not in main. I don't fully agree with FQA due to a bug not yet in main, I think it opens a less then ideal precedent, though I understand that it makes sense in this instance. Here is what I did: 1) Fully rebased the branch on current main 2) Made it depend on bug 41167 3) Added the patch: https://github.com/openfifth/koha/commit/57c5b64943ab72a744c2f9fcadf4c8b21e9d0b31 > > 1. There are no cypress tests > > Yes, there are. See t/cypress/integration/SIP2. No idea how I missed that! > > 4. If I upgrade I do have stuffs in the DB (picked from > > /etc/koha/sites/kohadev/SIPconfig.xml) but for new installs the table are > > not populated. > > Maybe we need to patch miscdev(?) > > Out of scope here? AFAIK the 'dummy' existing config in SIPconfig.xml served > only as documentation for all the possible configuration values for accounts > + institutions + syspref overrides (which is what is migrated here). Server > params and listeners are still read from the SIPconfig.xml file (as per > DCook's original QA requirements). It's a regression for ktd: we are expecting to have a working configuration. See https://gitlab.com/koha-community/koha-misc4dev/-/issues/35 and the related merge request "Issue #35 - Make SIP work out of the box" > > 7 Don't we want a config flag to turn off the ability to edit from the UI? > > I don't think so? The ability to access and work with the module is already > governed by the new 'sip2' permission. Yes, but prior to this change only sysops were able to edit it. I would like Kyle and Tomas opinion on this one. Thanks for the other fixes! (In reply to Jonathan Druart from comment #62) > > Out of scope here? AFAIK the 'dummy' existing config in SIPconfig.xml served > > only as documentation for all the possible configuration values for accounts > > + institutions + syspref overrides (which is what is migrated here). Server > > params and listeners are still read from the SIPconfig.xml file (as per > > DCook's original QA requirements). > > It's a regression for ktd: we are expecting to have a working configuration. > See https://gitlab.com/koha-community/koha-misc4dev/-/issues/35 and the > related merge request "Issue #35 - Make SIP work out of the box" C4::SIP::Sip::Configuration::get_configuration only fetches accounts, institutions and syspref_overrides from the database if any exist (in the database). Otherwise it falls back to the old way: reading that data from SIPconfig.xml. This was purposely done to have a smooth transition i.e. if for any reason the data is not in the database, reading from the SIPconfig.xml is still the fallback (until full deprecation in the future(?)). I'm happy to help/follow-up with whatever is needed, though I've not worked on misc4dev before and would appreciate guidance, would it be a new sip_data.sql file under installer/data/mysql/mandatory? Thank you. Cypress tests failing after following QA feedback entries. Patch updating cypress tests pushed: https://github.com/openfifth/koha/commit/2201bb08a1b644992f840e71e2578efe5e31665d (In reply to Pedro Amorim from comment #63) > (In reply to Jonathan Druart from comment #62) > > > Out of scope here? AFAIK the 'dummy' existing config in SIPconfig.xml served > > > only as documentation for all the possible configuration values for accounts > > > + institutions + syspref overrides (which is what is migrated here). Server > > > params and listeners are still read from the SIPconfig.xml file (as per > > > DCook's original QA requirements). > > > > It's a regression for ktd: we are expecting to have a working configuration. > > See https://gitlab.com/koha-community/koha-misc4dev/-/issues/35 and the > > related merge request "Issue #35 - Make SIP work out of the box" > > C4::SIP::Sip::Configuration::get_configuration only fetches accounts, > institutions and syspref_overrides from the database if any exist (in the > database). Otherwise it falls back to the old way: reading that data from > SIPconfig.xml. This was purposely done to have a smooth transition i.e. if > for any reason the data is not in the database, reading from the > SIPconfig.xml is still the fallback (until full deprecation in the > future(?)). > > I'm happy to help/follow-up with whatever is needed, though I've not worked > on misc4dev before and would appreciate guidance, would it be a new > sip_data.sql file under installer/data/mysql/mandatory? Thank you. Open an issue there and we will deal with that later. It seems that we could detect if the dbrev exists, if it exists we can run it and it will populate the DB. (In reply to Pedro Amorim from comment #61) > > (In reply to Jonathan Druart from comment #59) > > > 11. Missing changes in etc/koha-httpd.conf? > > > > I haven't checked all the places Jonathan mentioned but I have checked this > > one and in light of bug 41167, I'm marking this as Failed QA > > Thank you for bringing my attention to bug 41167, I was not aware of it. > However bug 41167 is not in main. I don't fully agree with FQA due to a bug > not yet in main, I think it opens a less then ideal precedent, though I > understand that it makes sense in this instance. Here is what I did: > 1) Fully rebased the branch on current main > 2) Made it depend on bug 41167 > 3) Added the patch: > https://github.com/openfifth/koha/commit/ > 57c5b64943ab72a744c2f9fcadf4c8b21e9d0b31 That's a good point. Initially I wasn't thinking of depending on bug 41167 but rather just adding the required 1 line to etc/koha-httpd.conf, but then that probably would've made for a merge conflict. Just bad timing, but thanks for taking care of it! |