|
Description
Jonathan Druart
2019-12-03 19:44:15 UTC
Created attachment 97120 [details] [review] Bug 24163: Allow to define CSV profile for late orders export This new enhancement adds the ability to create a CSV profile for the late orders export. Prior to this, the CSV profile was hardcoded in a template and not editable (unless you modify the .tt file of course). The main difficulty of the request was to make the multiple claims dates accessible from the CSV profile. So far we only accept columns from the database. However we would like to use the Koha::Acquisition::Order object to make things easier (it had a ->claims method to access the different claims). To acchieve this, we are going to accept the TT syntax for that CSV profile. It means that only CSV profiles created for 'late_orders' will have the capability to use the TT syntax (can be extended later of course if needed). The alternative was to use specific placeholders, like %claims_count%, %claimed_date%, but it sounded more powerful and flexible to allow the TT syntax instead. Note that the former export (template based) still exists and is the default option. Test plan: 0. Apply the patches and execute the update database entry 1. Create some orders, close the basket 2. Claim some of the orders 3. Note the new button at the bottom of the table that have several entries. => The first entry will generate a CSV file using the previous method => The second entry will generate it using the new CSV profile, note that the number of claims as well as the different claims date are displayed. 4. Bonus point: Edit the CSV profile (Home › Tools › CSV export profiles) that is named "Late orders (csv profile)", or create a new one, and modify it. Export the late orders using this CSV profile and confirm that the generated CSV file is the one you expect. Created attachment 97121 [details] [review] Bug 24163: Add the csv into the DB - as an example Created attachment 97122 [details] [review] Bug 24163: Allow to define CSV profile for late orders export This new enhancement adds the ability to create a CSV profile for the late orders export. Prior to this, the CSV profile was hardcoded in a template and not editable (unless you modify the .tt file of course). The main difficulty of the request was to make the multiple claims dates accessible from the CSV profile. So far we only accept columns from the database. However we would like to use the Koha::Acquisition::Order object to make things easier (it had a ->claims method to access the different claims). To acchieve this, we are going to accept the TT syntax for that CSV profile. It means that only CSV profiles created for 'late_orders' will have the capability to use the TT syntax (can be extended later of course if needed). The alternative was to use specific placeholders, like %claims_count%, %claimed_date%, but it sounded more powerful and flexible to allow the TT syntax instead. Note that the former export (template based) still exists and is the default option. Test plan: 0. Apply the patches and execute the update database entry 1. Create some orders, close the basket 2. Claim some of the orders 3. Note the new button at the bottom of the table that have several entries. => The first entry will generate a CSV file using the previous method => The second entry will generate it using the new CSV profile, note that the number of claims as well as the different claims date are displayed. 4. Bonus point: Edit the CSV profile (Home › Tools › CSV export profiles) that is named "Late orders (csv profile)", or create a new one, and modify it. Export the late orders using this CSV profile and confirm that the generated CSV file is the one you expect. Sponsored-by: Institute of Technology Tralee Created attachment 97123 [details] [review] Bug 24163: Add the csv into the DB - as an example Sponsored-by: Institute of Technology Tralee Created attachment 98691 [details] [review] Bug 24163: Allow to define CSV profile for late orders export This new enhancement adds the ability to create a CSV profile for the late orders export. Prior to this, the CSV profile was hardcoded in a template and not editable (unless you modify the .tt file of course). The main difficulty of the request was to make the multiple claims dates accessible from the CSV profile. So far we only accept columns from the database. However we would like to use the Koha::Acquisition::Order object to make things easier (it had a ->claims method to access the different claims). To acchieve this, we are going to accept the TT syntax for that CSV profile. It means that only CSV profiles created for 'late_orders' will have the capability to use the TT syntax (can be extended later of course if needed). The alternative was to use specific placeholders, like %claims_count%, %claimed_date%, but it sounded more powerful and flexible to allow the TT syntax instead. Note that the former export (template based) still exists and is the default option. Test plan: 0. Apply the patches and execute the update database entry 1. Create some orders, close the basket 2. Claim some of the orders 3. Note the new button at the bottom of the table that have several entries. => The first entry will generate a CSV file using the previous method => The second entry will generate it using the new CSV profile, note that the number of claims as well as the different claims date are displayed. 4. Bonus point: Edit the CSV profile (Home › Tools › CSV export profiles) that is named "Late orders (csv profile)", or create a new one, and modify it. Export the late orders using this CSV profile and confirm that the generated CSV file is the one you expect. Sponsored-by: Institute of Technology Tralee Created attachment 98692 [details] [review] Bug 24163: Add the csv into the DB - as an example Sponsored-by: Institute of Technology Tralee Created attachment 102185 [details] [review] Bug 24163: Allow to define CSV profile for late orders export This new enhancement adds the ability to create a CSV profile for the late orders export. Prior to this, the CSV profile was hardcoded in a template and not editable (unless you modify the .tt file of course). The main difficulty of the request was to make the multiple claims dates accessible from the CSV profile. So far we only accept columns from the database. However we would like to use the Koha::Acquisition::Order object to make things easier (it had a ->claims method to access the different claims). To acchieve this, we are going to accept the TT syntax for that CSV profile. It means that only CSV profiles created for 'late_orders' will have the capability to use the TT syntax (can be extended later of course if needed). The alternative was to use specific placeholders, like %claims_count%, %claimed_date%, but it sounded more powerful and flexible to allow the TT syntax instead. Note that the former export (template based) still exists and is the default option. Test plan: 0. Apply the patches and execute the update database entry 1. Create some orders, close the basket 2. Claim some of the orders 3. Note the new button at the bottom of the table that have several entries. => The first entry will generate a CSV file using the previous method => The second entry will generate it using the new CSV profile, note that the number of claims as well as the different claims date are displayed. 4. Bonus point: Edit the CSV profile (Home › Tools › CSV export profiles) that is named "Late orders (csv profile)", or create a new one, and modify it. Export the late orders using this CSV profile and confirm that the generated CSV file is the one you expect. Sponsored-by: Institute of Technology Tralee Created attachment 102186 [details] [review] Bug 24163: Add the csv into the DB - as an example Sponsored-by: Institute of Technology Tralee Created attachment 102257 [details] [review] Bug 24163: Allow to define CSV profile for late orders export This new enhancement adds the ability to create a CSV profile for the late orders export. Prior to this, the CSV profile was hardcoded in a template and not editable (unless you modify the .tt file of course). The main difficulty of the request was to make the multiple claims dates accessible from the CSV profile. So far we only accept columns from the database. However we would like to use the Koha::Acquisition::Order object to make things easier (it had a ->claims method to access the different claims). To acchieve this, we are going to accept the TT syntax for that CSV profile. It means that only CSV profiles created for 'late_orders' will have the capability to use the TT syntax (can be extended later of course if needed). The alternative was to use specific placeholders, like %claims_count%, %claimed_date%, but it sounded more powerful and flexible to allow the TT syntax instead. Note that the former export (template based) still exists and is the default option. Test plan: 0. Apply the patches and execute the update database entry 1. Create some orders, close the basket 2. Claim some of the orders 3. Note the new button at the bottom of the table that have several entries. => The first entry will generate a CSV file using the previous method => The second entry will generate it using the new CSV profile, note that the number of claims as well as the different claims date are displayed. 4. Bonus point: Edit the CSV profile (Home › Tools › CSV export profiles) that is named "Late orders (csv profile)", or create a new one, and modify it. Export the late orders using this CSV profile and confirm that the generated CSV file is the one you expect. Sponsored-by: Institute of Technology Tralee Signed-off-by: David Nind <david@davidnind.com> Created attachment 102258 [details] [review] Bug 24163: Add the csv into the DB - as an example Sponsored-by: Institute of Technology Tralee Signed-off-by: David Nind <david@davidnind.com> Created attachment 103585 [details] [review] Bug 24163: Allow to define CSV profile for late orders export This new enhancement adds the ability to create a CSV profile for the late orders export. Prior to this, the CSV profile was hardcoded in a template and not editable (unless you modify the .tt file of course). The main difficulty of the request was to make the multiple claims dates accessible from the CSV profile. So far we only accept columns from the database. However we would like to use the Koha::Acquisition::Order object to make things easier (it had a ->claims method to access the different claims). To acchieve this, we are going to accept the TT syntax for that CSV profile. It means that only CSV profiles created for 'late_orders' will have the capability to use the TT syntax (can be extended later of course if needed). The alternative was to use specific placeholders, like %claims_count%, %claimed_date%, but it sounded more powerful and flexible to allow the TT syntax instead. Note that the former export (template based) still exists and is the default option. Test plan: 0. Apply the patches and execute the update database entry 1. Create some orders, close the basket 2. Claim some of the orders 3. Note the new button at the bottom of the table that have several entries. => The first entry will generate a CSV file using the previous method => The second entry will generate it using the new CSV profile, note that the number of claims as well as the different claims date are displayed. 4. Bonus point: Edit the CSV profile (Home › Tools › CSV export profiles) that is named "Late orders (csv profile)", or create a new one, and modify it. Export the late orders using this CSV profile and confirm that the generated CSV file is the one you expect. Sponsored-by: Institute of Technology Tralee Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Angela O'Connor Desmond <angela.oconnordesmond@staff.ittralee.ie> Created attachment 103586 [details] [review] Bug 24163: Add the csv into the DB - as an example Sponsored-by: Institute of Technology Tralee Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Angela O'Connor Desmond <angela.oconnordesmond@staff.ittralee.ie> 1) Typo and rephrase + <p>For late orders exports you must provide a Template Toolkit syntax that will generate the whole CSV file..</p> I suggest a sight change here: For exporting late orders you must provide a profile in Template Toolkit syntax to generate the CSV file. Late orders (csv profile) => Late orders (CSV profile) default CSV export for late orders => Default ... 2) Other installers Please add the profile to the other installer sample files as well. 3) Default When you look at the pull down of the split button, it says "Default". Is the default different to the new template in significant ways? Wondering if we need to keep it or could replace it with the CSV profile. (question) 4) Diacritics / encoding I was testing with our sample record: Typical girls? : Author: Howe, Zoë Street. The default is ok, but in the CSV profile export Zoë is broken. Created attachment 104192 [details] [review] Bug 24163: (QA follow-up) Fix wordings Created attachment 104193 [details] [review] Bug 24163: Add the new CSV profile to other installer files export_format is not populated for other languages apparently... Created attachment 104194 [details] [review] Bug 24163: Specify the UTF-8 charset in the header To avoid encoding issues (In reply to Katrin Fischer from comment #13) > 1) Typo and rephrase > > + <p>For late orders exports you must provide a Template > Toolkit syntax that will generate the whole CSV file..</p> > > I suggest a sight change here: > For exporting late orders you must provide a profile in Template Toolkit > syntax to generate the CSV file. > > Late orders (csv profile) => Late orders (CSV profile) > default CSV export for late orders => Default ... Fixed! > 2) Other installers > > Please add the profile to the other installer sample files as well. Done! > 3) Default > > When you look at the pull down of the split button, it says "Default". Is > the default different to the new template in significant ways? Wondering if > we need to keep it or could replace it with the CSV profile. (question) Yes it's different, the "Default" has: ORDER DATE;ESTIMATED DELIVERY DATE;VENDOR;INFORMATION;TOTAL COST;BASKET;CLAIMS COUNT;CLAIMED DATE;INTERNAL NOTE;VENDOR NOTE;ISBN And has an additional line at the bottom: ;;"Total orders in late";1 When the new one has: Title,Author,Publication year,ISBN,Quantity,Number of claims I thought it was better to keep the default one (for those who use it), but we can remove it if needed. > 4) Diacritics / encoding > > I was testing with our sample record: Typical girls? : > Author: Howe, Zoë Street. > > The default is ok, but in the CSV profile export Zoë is broken. Good catch, fixed! Hi Jonathan, can you please retest? I am still seeing the same issue with diacritics and the new CSV profiles. Checked twice I have the new patches, restart_all and the Default CSV is working correct. When I download the CSV and change the encoding form UTF-8 to Latin 3 (ISO 8859-3 ) in the import screen from LibreOffice, Zoë will display correctly. Created attachment 104252 [details] [review] Bug 24163: UTF-8 encode the output Specify the UTF-8 charset in the header and Encode the output to avoid encoding issues Sorry, just found local changes I had not embedded in the last patch! Created attachment 104337 [details] [review] Bug 24163: Allow to define CSV profile for late orders export This new enhancement adds the ability to create a CSV profile for the late orders export. Prior to this, the CSV profile was hardcoded in a template and not editable (unless you modify the .tt file of course). The main difficulty of the request was to make the multiple claims dates accessible from the CSV profile. So far we only accept columns from the database. However we would like to use the Koha::Acquisition::Order object to make things easier (it had a ->claims method to access the different claims). To acchieve this, we are going to accept the TT syntax for that CSV profile. It means that only CSV profiles created for 'late_orders' will have the capability to use the TT syntax (can be extended later of course if needed). The alternative was to use specific placeholders, like %claims_count%, %claimed_date%, but it sounded more powerful and flexible to allow the TT syntax instead. Note that the former export (template based) still exists and is the default option. Test plan: 0. Apply the patches and execute the update database entry 1. Create some orders, close the basket 2. Claim some of the orders 3. Note the new button at the bottom of the table that have several entries. => The first entry will generate a CSV file using the previous method => The second entry will generate it using the new CSV profile, note that the number of claims as well as the different claims date are displayed. 4. Bonus point: Edit the CSV profile (Home › Tools › CSV export profiles) that is named "Late orders (csv profile)", or create a new one, and modify it. Export the late orders using this CSV profile and confirm that the generated CSV file is the one you expect. Sponsored-by: Institute of Technology Tralee Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Angela O'Connor Desmond <angela.oconnordesmond@staff.ittralee.ie> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Created attachment 104338 [details] [review] Bug 24163: Add the csv into the DB - as an example Sponsored-by: Institute of Technology Tralee Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Angela O'Connor Desmond <angela.oconnordesmond@staff.ittralee.ie> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Created attachment 104339 [details] [review] Bug 24163: (QA follow-up) Fix wordings Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Created attachment 104340 [details] [review] Bug 24163: Add the new CSV profile to other installer files export_format is not populated for other languages apparently... Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Created attachment 104341 [details] [review] Bug 24163: UTF-8 encode the output Specify the UTF-8 charset in the header and Encode the output to avoid encoding issues Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Nice work everyone! Pushed to master for 20.05 not backported to 19.11.x |