Bug 32706

Summary: User search problem caused by branchcode errors
Product: Koha Reporter: Hammat wele <hammat.wele>
Component: PatronsAssignee: Hammat wele <hammat.wele>
Status: CLOSED WONTFIX QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: caroline.cyr-la-rose, gmcharlt, jonathan.druart, kyle.m.hall, m.de.rooy, philippe.blouin
Version: master   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32517
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33539
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 32706: User search problem caused by branchcode errors
Bug 32706: User search problem caused by branchcode errors

Description Hammat wele 2023-01-23 22:00:45 UTC
Searching users alphabetically fails when there are two similar branchcode types for the same value in the database: one written in uppercase and the second in lowercase

Prerequisites
1. Create a new library with library code written in uppercase
1.1. Go to Administration > Libraries
1.2. Click on New library
1.3. Fill in the form
Library code = uppercase code (for example TST )
Name = type a name (for example Test category )
1.4. Save
2. Create several patrons in the TST library
     2.1. Go to the patron module
     2.2. Click on “new patron”
     2.3 Fill the form
         Surname, First name, Card number, Library
         (Choose the same 'TST' library for all users that will be created)
     2.4. Save
     Create other users (~10 )
3. Export users
     3.1. Go to the Report module
     3.2. Click on Create from SQL
     3.3. Type a Report name and paste the following query to the SQL field:
SELECT
    borrowers.cardnumber,
    borrowers.surname,
    borrowers.firstname,
    borrowers.categorycode,
    borrowers.branchcode,
    borrowers.dateexpiry
FROM
    borrowers
WHERE
       borrowers.branchcode= <<branchcode|branches>>
ORDER BY borrowers.cardnumber asc
4. Click on Save report then on 'Run report'
5. Choose the branch from step 1.3 and Click on Run the report
5. Download report results
6. Import user folders with lowercase branchcode
         6.1. Open the file from step 5
         6.2. Edit data
         6.3. Remove half of the users from the file
         6.4. In the "branchcode" column, replace each value with a lowercase branchcode (for example 'tst')
         6.5. Save the file in .csv format
         6.6. Go to Tools > Patron imports
         6.7. upload the file
         6.8. Click on Overwrite the existing one with this
         6.9. Click on Replace only included patron attributes
         6.10. Click on “Import”


Test plan
1. Go to the Patron module
2. Click on a letter of the alphabet 'different' from the users created in  step 2.2
     --> Koha opens the results
3. Choose a last name letter from one of the users created in  step 2.2
     --> Koha freezes and it leaves the message “Processing”
Comment 1 Katrin Fischer 2023-01-23 22:16:19 UTC
Hm, this sounds similar to bug 32517 that dealt with patron category codes.
Comment 2 Hammat wele 2023-01-23 22:34:58 UTC
Yes it's quite similar, in this case it's the branchcode instead of the categorycode we facing the same problem with the branchcode too but the the js error is «Uncaught TypeError: libraries_map[data] is undefined»
Comment 3 Hammat wele 2023-01-23 22:39:54 UTC
Created attachment 145593 [details] [review]
Bug 32706: User search problem caused by branchcode errors

Searching users alphabetically fails when there are two similar branchcode types for the same value in the database: one written in uppercase and the second in lowercase.

Prerequisites
1. Create a new library with library code written in uppercase
1.1. Go to Administration > Libraries
1.2. Click on New library
1.3. Fill in the form
Library code = uppercase code (for example TST )
Name = type a name (for example Test category )
1.4. Save
2. Create several patrons in the TST library
     2.1. Go to the patron module
     2.2. Click on “new patron”
     2.3 Fill the form
         Surname, First name, Card number, Library
         (Choose the same 'TST' library for all users that will be created)
     2.4. Save
     Create other users (~10 )
3. Export users
     3.1. Go to the Report module
     3.2. Click on Create from SQL
     3.3. Type a Report name and paste the following query to the SQL field:
SELECT
    borrowers.cardnumber,
    borrowers.surname,
    borrowers.firstname,
    borrowers.categorycode,
    borrowers.branchcode,
    borrowers.dateexpiry
FROM
    borrowers
WHERE
       borrowers.branchcode= <<branchcode|branches>>
ORDER BY borrowers.cardnumber asc
4. Click on Save report then on 'Run report'
5. Choose the branch from step 1.3 and Click on Run the report
5. Download report results
6. Import user folders with lowercase branchcode
         6.1. Open the file from step 5
         6.2. Edit data
         6.3. Remove half of the users from the file
         6.4. In the 'branchcode' column, replace each value with a lowercase branchcode (for example 'tst')
         6.5. Save the file in .csv format
         6.6. Go to Tools > Patron imports
         6.7. upload the file
         6.8. Click on Overwrite the existing one with this
         6.9. Click on Replace only included patron attributes
         6.10. Click on “Import”

Test plan
1. Go to the Patron module
2. Click on a letter of the alphabet 'different' from the users created in  step 2.2
     --> Koha opens the results
3. Choose a last name letter from one of the users created in  step 2.2
     --> Koha freezes and it leaves the message 'Processing'
4. Apply the patch
5. Repeat step 1,2,3
     --> the results are displayed correctly
Comment 4 David Nind 2023-01-24 21:48:46 UTC
Created attachment 145636 [details] [review]
Bug 32706: User search problem caused by branchcode errors

Searching users alphabetically fails when there are two similar branchcode types for the same value in the database: one written in uppercase and the second in lowercase.

Prerequisites
1. Create a new library with library code written in uppercase
1.1. Go to Administration > Libraries
1.2. Click on New library
1.3. Fill in the form
Library code = uppercase code (for example TST )
Name = type a name (for example Test category )
1.4. Save
2. Create several patrons in the TST library
     2.1. Go to the patron module
     2.2. Click on “new patron”
     2.3 Fill the form
         Surname, First name, Card number, Library
         (Choose the same 'TST' library for all users that will be created)
     2.4. Save
     Create other users (~10 )
3. Export users
     3.1. Go to the Report module
     3.2. Click on Create from SQL
     3.3. Type a Report name and paste the following query to the SQL field:
SELECT
    borrowers.cardnumber,
    borrowers.surname,
    borrowers.firstname,
    borrowers.categorycode,
    borrowers.branchcode,
    borrowers.dateexpiry
FROM
    borrowers
WHERE
       borrowers.branchcode= <<branchcode|branches>>
ORDER BY borrowers.cardnumber asc
4. Click on Save report then on 'Run report'
5. Choose the branch from step 1.3 and Click on Run the report
5. Download report results
6. Import user folders with lowercase branchcode
         6.1. Open the file from step 5
         6.2. Edit data
         6.3. Remove half of the users from the file
         6.4. In the 'branchcode' column, replace each value with a lowercase branchcode (for example 'tst')
         6.5. Save the file in .csv format
         6.6. Go to Tools > Patron imports
         6.7. upload the file
         6.8. Click on Overwrite the existing one with this
         6.9. Click on Replace only included patron attributes
         6.10. Click on “Import”

Test plan
1. Go to the Patron module
2. Click on a letter of the alphabet 'different' from the users created in  step 2.2
     --> Koha opens the results
3. Choose a last name letter from one of the users created in  step 2.2
     --> Koha freezes and it leaves the message 'Processing'
4. Apply the patch
5. Repeat step 1,2,3
     --> the results are displayed correctly

Signed-off-by: David Nind <david@davidnind.com>
Comment 5 Jonathan Druart 2023-02-23 10:52:03 UTC
You are inserting incorrect data then expect the software to behave correctly.

To me this is a migration problem, not something we should fix in Koha.
Comment 6 Jonathan Druart 2023-02-23 10:52:34 UTC
If you want to prevent it, then fix the import tool.
Comment 7 Jonathan Druart 2023-02-23 10:52:51 UTC
Letting open to let others give their opinion.
Comment 8 Katrin Fischer 2023-02-23 10:54:52 UTC
(In reply to Jonathan Druart from comment #7)
> Letting open to let others give their opinion.

This is the same than bug 32517 for category codes. I argued similarly there, but I think we should not push one and block the other.
Comment 9 Marcel de Rooy 2023-04-14 07:21:58 UTC
Yes, we should fix import.
Comment 10 Marcel de Rooy 2023-04-14 07:23:39 UTC
(In reply to Katrin Fischer from comment #8)
> (In reply to Jonathan Druart from comment #7)
> > Letting open to let others give their opinion.
> 
> This is the same than bug 32517 for category codes. I argued similarly
> there, but I think we should not push one and block the other.

Understand the point. Increasing insight?
Comment 11 Katrin Fischer 2023-04-15 20:21:21 UTC
Thinking more about it, I think this type of fix is a bit dangerous as we cannot tell what other hidden side effects wrong branchcodes will have, I am pretty sure there will be problems in other areas.

I've filed a bug to add a capitalization check to the inconsistencies script:
Bug 33539