Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing
Summary: Add script to allow developers to quickly generate large amounts of data for ...
Status: Signed Off
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-07-24 12:11 UTC by Kyle M Hall
Modified: 2024-07-25 10:59 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing (2.99 KB, patch)
2024-07-24 12:17 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing (2.96 KB, patch)
2024-07-24 12:55 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing (3.02 KB, patch)
2024-07-24 13:15 UTC, Pedro Amorim
Details | Diff | Splinter Review
Bug 37448: Make use of build_sample_biblio and build_sample_item (1.72 KB, patch)
2024-07-24 13:15 UTC, Pedro Amorim
Details | Diff | Splinter Review
Bug 37448: Make use of build_sample_biblio and build_sample_item (1.78 KB, patch)
2024-07-24 13:35 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 37448: Make use of build_sample_biblio and build_sample_item (1.79 KB, patch)
2024-07-24 13:41 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 37448: Add build_sample_ill_request (2.29 KB, patch)
2024-07-24 13:49 UTC, Pedro Amorim
Details | Diff | Splinter Review
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing (3.02 KB, patch)
2024-07-25 10:58 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 37448: Make use of build_sample_biblio and build_sample_item (1.79 KB, patch)
2024-07-25 10:59 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 37448: Add build_sample_ill_request (2.66 KB, patch)
2024-07-25 10:59 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 37448: (QA follow-up) Tidy script (1.26 KB, patch)
2024-07-25 10:59 UTC, Kyle M Hall
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Kyle M Hall 2024-07-24 12:11:47 UTC

    
Comment 1 Kyle M Hall 2024-07-24 12:17:48 UTC
Created attachment 169472 [details] [review]
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing

Test Plan:
1) Apply this patch
2) Run: misc/devel/create_test_data.pl -n 99 -s Borrower -d surname=Hall  -d zipcode=111111
3) Search patrons' for the name "Hall"
4) Note there are 99 Hall's in your results!

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 2 Kyle M Hall 2024-07-24 12:55:24 UTC
Created attachment 169474 [details] [review]
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing

Test Plan:
1) Apply this patch
2) Run: misc/devel/create_test_data.pl -n 99 -s Borrower -d surname=Hall  -d zipcode=111111
3) Search patrons' for the name "Hall"
4) Note there are 99 Hall's in your results!

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 3 Pedro Amorim 2024-07-24 13:15:01 UTC
Created attachment 169486 [details] [review]
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing

Test Plan:
1) Apply this patch
2) Run: misc/devel/create_test_data.pl -n 99 -s Borrower -d surname=Hall  -d zipcode=111111
3) Search patrons' for the name "Hall"
4) Note there are 99 Hall's in your results!

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Comment 4 Pedro Amorim 2024-07-24 13:15:04 UTC
Created attachment 169487 [details] [review]
Bug 37448: Make use of build_sample_biblio and build_sample_item

The script is brilliant, but for biblios and items we should make use of build_sample_biblio and build_sample_item or this data does not get indexed + linked tables rows get missed

To test, before this patch, run:
1) misc/devel/create_test_data.pl -n 5 -s Biblio -d title=Test
  Notice the 'Test' biblio is created on the database, but doesnt show on searches, and accessing it directly through URL throws a 500 error (because metadata does not exist for the biblio)
2) Apply this patch. Repeat the step above. Notice it now shows on searches and visiting the biblio directly shows no errors

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Comment 5 Pedro Amorim 2024-07-24 13:17:08 UTC
Side info:
Enabling ILL and installing FreeForm by running:
  bash <(curl -s https://raw.githubusercontent.com/ammopt/koha-ill-dev/master/start-ill-dev.sh)

If we then run:
  misc/devel/create_test_data.pl -n 99 -s Illrequest -d backend=FreeForm

It works beautifully, and although not an exact 1 to 1 to what I did at
https://github.com/ammopt/koha-ill-dev/blob/master/lib/KohaFactory/ILL.pm

This way using the test builder instead is much more elegant and practical. Thanks Kyle this is awesome.
Comment 6 Kyle M Hall 2024-07-24 13:35:52 UTC
Created attachment 169488 [details] [review]
Bug 37448: Make use of build_sample_biblio and build_sample_item

The script is brilliant, but for biblios and items we should make use of build_sample_biblio and build_sample_item or this data does not get indexed + linked tables rows get missed

To test, before this patch, run:
1) misc/devel/create_test_data.pl -n 5 -s Biblio -d title=Test
  Notice the 'Test' biblio is created on the database, but doesnt show on searches, and accessing it directly through URL throws a 500 error (because metadata does not exist for the biblio)
2) Apply this patch. Repeat the step above. Notice it now shows on searches and visiting the biblio directly shows no errors

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 7 Kyle M Hall 2024-07-24 13:41:35 UTC
Created attachment 169489 [details] [review]
Bug 37448: Make use of build_sample_biblio and build_sample_item

The script is brilliant, but for biblios and items we should make use of build_sample_biblio and build_sample_item or this data does not get indexed + linked tables rows get missed

To test, before this patch, run:
1) misc/devel/create_test_data.pl -n 5 -s Biblio -d title=Test
  Notice the 'Test' biblio is created on the database, but doesnt show on searches, and accessing it directly through URL throws a 500 error (because metadata does not exist for the biblio)
2) Apply this patch. Repeat the step above. Notice it now shows on searches and visiting the biblio directly shows no errors

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 8 Kyle M Hall 2024-07-24 13:42:00 UTC
Thanks for the excellent followup Pedro!
Comment 9 Pedro Amorim 2024-07-24 13:49:13 UTC
Created attachment 169490 [details] [review]
Bug 37448: Add build_sample_ill_request

The linked biblio_id of a generated test ILL request needs to be created by build_sample_biblio->AddBiblio.
Or else the related biblio is created by just 'build' and is not indexed + its related metadata is missing.

To test:
1) Enabling ILL and installing FreeForm by running:
  bash <(curl -s https://raw.githubusercontent.com/ammopt/koha-ill-dev/master/start-ill-dev.sh)
2) Run the script for ILL requests:
  misc/devel/create_test_data.pl -n 5 -s Illrequest -d backend=FreeForm
3) Visit the ILLModule:
  http://<intra_url>/cgi-bin/koha/ill/ill-requests.pl
4) Notice it loads 5 test ILL requests correctly

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Comment 10 Pedro Amorim 2024-07-24 13:50:06 UTC
Apologies for the additional follow-up, but upon further testing this is required for ILL requests
I want to make use of this script for ILL testing (!).
Comment 11 Kyle M Hall 2024-07-25 10:58:54 UTC
Created attachment 169550 [details] [review]
Bug 37448 - Add script to allow developers to quickly generate large amounts of data for development and testing

Test Plan:
1) Apply this patch
2) Run: misc/devel/create_test_data.pl -n 99 -s Borrower -d surname=Hall  -d zipcode=111111
3) Search patrons' for the name "Hall"
4) Note there are 99 Hall's in your results!

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Comment 12 Kyle M Hall 2024-07-25 10:59:02 UTC
Created attachment 169551 [details] [review]
Bug 37448: Make use of build_sample_biblio and build_sample_item

The script is brilliant, but for biblios and items we should make use of build_sample_biblio and build_sample_item or this data does not get indexed + linked tables rows get missed

To test, before this patch, run:
1) misc/devel/create_test_data.pl -n 5 -s Biblio -d title=Test
  Notice the 'Test' biblio is created on the database, but doesnt show on searches, and accessing it directly through URL throws a 500 error (because metadata does not exist for the biblio)
2) Apply this patch. Repeat the step above. Notice it now shows on searches and visiting the biblio directly shows no errors

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 13 Kyle M Hall 2024-07-25 10:59:04 UTC
Created attachment 169552 [details] [review]
Bug 37448: Add build_sample_ill_request

The linked biblio_id of a generated test ILL request needs to be created by build_sample_biblio->AddBiblio.
Or else the related biblio is created by just 'build' and is not indexed + its related metadata is missing.

To test:
1) Enabling ILL and installing FreeForm by running:
  bash <(curl -s https://raw.githubusercontent.com/ammopt/koha-ill-dev/master/start-ill-dev.sh)
2) Run the script for ILL requests:
  misc/devel/create_test_data.pl -n 5 -s Illrequest -d backend=FreeForm
3) Visit the ILLModule:
  http://<intra_url>/cgi-bin/koha/ill/ill-requests.pl
4) Notice it loads 5 test ILL requests correctly

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 14 Kyle M Hall 2024-07-25 10:59:06 UTC
Created attachment 169553 [details] [review]
Bug 37448: (QA follow-up) Tidy script

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>