Description
Eric Phetteplace
2021-10-13 21:50:25 UTC
Created attachment 126242 [details] [review] Bug 29238 - Cataloging cron jobs are not logged Here's an example patch. Is misc/migration_tools/remove_unused_authorities.pl maybe broken on master? Perl complained that DelAuthority() on line 102 was undefined, so I changed the "use" statement. But otherwise, the patch just adds a cronlogaction import and call when the script is run. Generally, `cronlogaction()` is called during dry runs like `remove_unused_authorities.pl --test` but not if the script is printing its usage information (e.g. `remove_unused_authorities.pl --help`). It'd be much more complicated to only log when data actually changes. Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty Created attachment 126243 [details]
Screenshot of the logs
We'd appreciate this tiny enhancement. I thought our linker cron wasn't running today because it wasn't in the log. Can we bump this up? Thanks! Eric, should this be at needs signoff? Yes, sorry, I'm not super familiar with Bugzilla. The patch doesn't apply anymore. There seems to be a couple of conflicts. Error while trying to run git bz apply 29238: Using index info to reconstruct a base tree... M misc/link_bibs_to_authorities.pl M misc/migration_tools/remove_unused_authorities.pl Falling back to patching base and 3-way merge... Auto-merging misc/migration_tools/remove_unused_authorities.pl CONFLICT (content): Merge conflict in misc/migration_tools/remove_unused_authorities.pl Auto-merging misc/link_bibs_to_authorities.pl CONFLICT (content): Merge conflict in misc/link_bibs_to_authorities.pl error: Failed to merge in the changes. Patch failed at 0001 Bug 29238 - Cataloging cron jobs are not logged Created attachment 172240 [details] [review] Bug 29238 - Cataloging cron jobs are not logged Updated patch based off master. The files were just touched in between, this patch doesn't do anything different. Weird that git merge wouldn't have been able to work through it. I get this error when attempting to apply this patch on my testing docker: Traceback (most recent call last): File "/usr/bin/git-bz", line 2834, in <module> applied = do_apply(bug_ref) ^^^^^^^^^^^^^^^^^ File "/usr/bin/git-bz", line 1918, in do_apply users = bug.server.get_xmlrpc_proxy().User.get({'names': [patch.attacher]})['users'] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1122, in __call__ return self.__send(self.__name, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1464, in __request response = self.__transport.request( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1166, in request return self.single_request(host, handler, request_body, verbose) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1178, in single_request http_conn = self.send_request(host, handler, request_body, verbose) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/bin/git-bz", line 1239, in send_request connection.putheader("Cookie", cookie) ^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'putheader' Created attachment 174260 [details] [review] Bug 29238 - Cataloging cron jobs are not logged Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty Sorry, I made the patch manually, but git bz doesn't seem to be able to apply manual patches. I've generated it with `git bz attach` so hopefully it applies now. Created attachment 174261 [details] [review] Bug 29238 - Cataloging cron jobs are not logged Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com> (In reply to Eric Phetteplace from comment #10) > Sorry, I made the patch manually, but git bz doesn't seem to be able to > apply manual patches. I've generated it with `git bz attach` so hopefully it > applies now. Thanks for writing it! (In reply to Eric Phetteplace from comment #10) > Sorry, I made the patch manually, but git bz doesn't seem to be able to > apply manual patches. I've generated it with `git bz attach` so hopefully it > applies now. I think it's not the manual bit, but somehow you got a very different header lines in the patch with your manual process than when using git bz (Author vs. From for example). git bz is better, but when you need to create a patch manually, you can use git format-patch, which should give you the same output. (In reply to Katrin Fischer from comment #13) > (In reply to Eric Phetteplace from comment #10) > > Sorry, I made the patch manually, but git bz doesn't seem to be able to > > apply manual patches. I've generated it with `git bz attach` so hopefully it > > applies now. > > I think it's not the manual bit, but somehow you got a very different header > lines in the patch with your manual process than when using git bz (Author > vs. From for example). git bz is better, but when you need to create a patch > manually, you can use git format-patch, which should give you the same > output. I used `git format-patch` for the original patch. Though when I go to `git format-patch -n HEAD^` on the bug's branch now, I do get a patch identical to the git bz one, with similar header lines. I'm not sure what I did wrong with my original, perhaps something in my git configuration, I made it a while ago. Anyways, I've learned to stick with git bz. WARN misc/cronjobs/merge_authorities.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 17, now: 18) OK misc/link_bibs_to_authorities.pl WARN misc/migration_tools/remove_unused_authorities.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 24, now: 25) Processing additional checks * Commit title does not start with 'Bug XXXXX: ' - 909111fdd9 Created attachment 174545 [details] [review] Bug 29238: Cataloging cron jobs are not logged Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> (In reply to Marcel de Rooy from comment #15) > Processing additional checks > > * Commit title does not start with 'Bug XXXXX: ' - 909111fdd9 Fixed commit title with ':' Ignoring the whitespace stuff Hm, looking at other uses of cronlogaction, aren't we missing parameters here? misc/cronjobs/automatic_item_modification_by_age.pl:use C4::Log qw( cronlogaction ); misc/cronjobs/automatic_item_modification_by_age.pl:cronlogaction({ info => $command_line_options }); misc/cronjobs/automatic_item_modification_by_age.pl:cronlogaction({ action => 'End', info => "COMPLETED" }); Can you please check? Created attachment 174951 [details] [review] Bug 29238: Cataloging cron jobs are not logged Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty (In reply to Katrin Fischer from comment #18) > Hm, looking at other uses of cronlogaction, aren't we missing parameters > here? > > misc/cronjobs/automatic_item_modification_by_age.pl:use C4::Log qw( > cronlogaction ); > misc/cronjobs/automatic_item_modification_by_age.pl:cronlogaction({ info => > $command_line_options }); > misc/cronjobs/automatic_item_modification_by_age.pl:cronlogaction({ action > => 'End', info => "COMPLETED" }); > > Can you please check? OK I added the command-line parameters to the info that's logged. In order to capture all the flags, $command_line_options has to come before GetOptions. Setting this back to Needs Signoff. Created attachment 174953 [details]
Screenshot of the logs
Created attachment 175040 [details] [review] Bug 29238: Cataloging cron jobs are not logged Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty squash Signed-off-by: David Nind <david@davidnind.com> WARN misc/cronjobs/merge_authorities.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 17, now: 19) WARN misc/link_bibs_to_authorities.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 36, now: 38) WARN misc/migration_tools/remove_unused_authorities.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 24, now: 26) Please fix inline. Created attachment 175414 [details] [review] Bug 29238: (follow up) tidy 3 scripts I couldn't get tidy to work on individual lines following the instructions for both vim and VSCode in the wiki, so instead I created a separate commit with the full scripts run through perltidy. Created attachment 175421 [details] [review] Bug 29238: Cataloging cron jobs are not logged Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty squash Signed-off-by: David Nind <david@davidnind.com> Created attachment 175422 [details] [review] Bug 29238: (follow up) tidy 3 scripts Signed-off-by: David Nind <david@davidnind.com> Thanks for trying, Eric. But given the number of small changes in the first patch, I would not recommend doing a full tidy now on the scripts. Please ask some help on IRC / Mattermost to get perltidy working? Created attachment 176884 [details] [review] Bug 29238: (follow up) tidy 3 scripts Comment on attachment 176884 [details] [review] Bug 29238: (follow up) tidy 3 scripts Obsolete d5787d96afb6ed2db02d7a211f2c08975d3da54f "tidy 3 scripts" patch. Since the roadmap contains a current sprint to tidy the whole codebase which will address the QA concerns, perhaps this could be merged without needing to tidy it? See the discussion on Matterhost from today (2025-01-21) in the ~Development channel. (In reply to Eric Phetteplace from comment #32) > Since the roadmap contains a current sprint to tidy the whole codebase which > will address the QA concerns, perhaps this could be merged without needing > to tidy it? See the discussion on Matterhost from today (2025-01-21) in the > ~Development channel. Back to SO Created attachment 177098 [details] [review] Bug 29238: Cataloging cron jobs are not logged Testing plan: 1. set CronjobLog to "Log" 2. Run the three cron jobs `perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm` 3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty 4. apply the patch 5. Repeat steps 2 & 3 but note the cron logs aren't empty squash Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Pushed for 25.05! Well done everyone, thank you! Nice work everyone! Pushed to 24.11.x for 24.11.04 |