Bug 37824 - Replace webpack with rspack for fun and profit
Summary: Replace webpack with rspack for fun and profit
Status: Pushed to main
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Paul Derscheid
QA Contact: Martin Renvoize
URL:
Keywords: additional_work_needed
Depends on:
Blocks: 37911
  Show dependency treegraph
 
Reported: 2024-09-03 15:32 UTC by Paul Derscheid
Modified: 2024-09-16 16:30 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00
Circulation function:


Attachments
Bug 37824: Replace webpack with rspack for fun and profit (67.97 KB, patch)
2024-09-03 15:41 UTC, Paul Derscheid
Details | Diff | Splinter Review
Bug 37824: Replace webpack with rspack for fun and profit (68.15 KB, patch)
2024-09-06 07:50 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 37824: Replace webpack with rspack for fun and profit (68.18 KB, patch)
2024-09-13 07:59 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 37824: (follow-up) Add rspack.config.js to Makefile.PL (689 bytes, patch)
2024-09-16 14:42 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 37824: (QA follow-up) Fix improper handling of jQuery and DataTables libraries by marking them as externals (1.72 KB, patch)
2024-09-16 16:30 UTC, Paul Derscheid
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Derscheid 2024-09-03 15:32:46 UTC
Webpack is slow and nobody likes it. It gives us a lot of flexibility, though (for example in comparison to rollup, which is pretty great).

Let's replace it with what's just reached v1 and is advertised as more or less a drop-in-replacement. 

Full Disclosure: rspack is a ByteDance project (MIT licensed, though).
Comment 1 Paul Derscheid 2024-09-03 15:41:33 UTC
Created attachment 170992 [details] [review]
Bug 37824: Replace webpack with rspack for fun and profit

I left the webpack.config.js as well as the devDependencies in place for now.
We can remove them in a follow-up patch on this bug after testing it out.

To test:
1) Run js:build and js:build:prod
2) Note the build time
3) Apply patch
4) Run yarn install
5) Run js:build and js:build:prod again
6) Note the much faster build time
7) Extra credit: take a look at the ERM or preservations module and make sure everything works as expected.
8) Extra credit: run the cypress tests.
9) Sign off or give your opinion
Comment 2 Marcel de Rooy 2024-09-06 07:50:48 UTC
Created attachment 171126 [details] [review]
Bug 37824: Replace webpack with rspack for fun and profit

I left the webpack.config.js as well as the devDependencies in place for now.
We can remove them in a follow-up patch on this bug after testing it out.

To test:
1) Run js:build and js:build:prod
2) Note the build time
3) Apply patch
4) Run yarn install
5) Run js:build and js:build:prod again
6) Note the much faster build time
7) Extra credit: take a look at the ERM or preservations module and make sure everything works as expected.
8) Extra credit: run the cypress tests.
9) Sign off or give your opinion

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Looks faster to me. Where webpack failed for me (Killed, error 137), this passed.
Comment 3 Paul Derscheid 2024-09-12 13:46:10 UTC
Concerning the production readiness of rspack, I'd just like to say that ByteDance themselves already use it in production. And I guess what works for a billion $ company should be stable enough for us?
Comment 4 Jonathan Druart 2024-09-12 13:55:43 UTC
I don't have the time to test it right now but if it's faster and considered stable we should switch!
Comment 5 Paul Derscheid 2024-09-12 14:08:25 UTC
There's a solid argument joubu made against mine:

ByteDance as well as the big testemonials:
- Microsoft
- Amazon
- Intuit
- Discord

have time and resources to adapt their codebases to changes in the build tool and we don't.

However: they aim to reach as close to 100% compatibility as possible with webpack, which to me means that:
- they can't do crazy stuff
- we can always fallback onto webpack, it's not going anywhere.

Therefore I think we can be a bit bleeding edge here ;)
Comment 6 Martin Renvoize 2024-09-13 06:11:48 UTC
I'll be taking this for a spin today. Now that I've researched the current state of these sorts of tools I'm happy for us to take the leap too. There's a clear back step should we need to take it, but the project itself does feel well regarded and supported so I don't predict we'll need to take such measures.

Assuming I get the same results as prior testers, and I can't see why I wouldn't, 8 think this is well worthwhile and likely to pass today.
Comment 7 Paul Derscheid 2024-09-13 07:19:17 UTC
Thanks for testing this Martin :)
Comment 8 Martin Renvoize 2024-09-13 07:59:50 UTC
Created attachment 171438 [details] [review]
Bug 37824: Replace webpack with rspack for fun and profit

I left the webpack.config.js as well as the devDependencies in place for now.
We can remove them in a follow-up patch on this bug after testing it out.

To test:
1) Run js:build and js:build:prod
2) Note the build time
3) Apply patch
4) Run yarn install
5) Run js:build and js:build:prod again
6) Note the much faster build time
7) Extra credit: take a look at the ERM or preservations module and make sure everything works as expected.
8) Extra credit: run the cypress tests.
9) Sign off or give your opinion

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Looks faster to me. Where webpack failed for me (Killed, error 137), this passed.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 9 Martin Renvoize 2024-09-13 08:00:25 UTC
I'm happy this is a solid improvement and worthwhile. All worked in testing very seamlessly.

Passing QA

Nice work Paul
Comment 10 Katrin Fischer 2024-09-16 08:58:13 UTC
Just wondering: do we need to do anything to make sure the yarn build step is done on other ktd, Jenkins, sandboxes?
Comment 11 Paul Derscheid 2024-09-16 09:00:46 UTC
Only of they directly call webpack which I hope is not the case.

If everything is just yarn *:* and the node_modules is up-to-date, it will just work (and better than before imo).
Comment 12 Katrin Fischer 2024-09-16 11:49:11 UTC
Pushed for 24.11!

Well done everyone, thank you!
Comment 13 Jonathan Druart 2024-09-16 12:39:08 UTC
Failure at ktd startup

koha-1       | $ rspack build --mode development
koha-1       | [rspack-cli] [Error:   × Rspack FS Error: Error: EACCES: permission denied, open '/kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/js/vue/dist/erm.js.map'
koha-1       |   │ 
koha-1       |   ╰─▶ Error: EACCES: permission denied, open '/kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/js/vue/dist/erm.js.map'
Comment 14 Jonathan Druart 2024-09-16 12:43:02 UTC
(In reply to Jonathan Druart from comment #13)
> Failure at ktd startup
> 
> koha-1       | $ rspack build --mode development
> koha-1       | [rspack-cli] [Error:   × Rspack FS Error: Error: EACCES:
> permission denied, open
> '/kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/js/vue/dist/erm.js.map'
> koha-1       |   │ 
> koha-1       |   ╰─▶ Error: EACCES: permission denied, open
> '/kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/js/vue/dist/erm.js.map'

I chowned my whole repo and now ktd is starting correctly.
Comment 15 Jonathan Druart 2024-09-16 12:48:09 UTC
Cypress tests failing since this change:

% yarn cypress run --spec t/cypress/integration/ERM/Agreements_spec.ts 

  1) Agreement CRUD operations                                                                            
       Delete agreement:                                                                                  
     Error: The following error originated from your application code, not from Cypress. It was caused by an unhandled promise rejection.
                                                                                                          
  > Cannot extend unknown button type: excelHtml5                                                         

When Cypress detects uncaught errors originating from your application it will automatically fail the current test.
                                                                                                          
This behavior is configurable, and you can choose to turn this off by listening to the `uncaught:exception` event.    
                                                     
https://on.cypress.io/uncaught-exception-from-application
Comment 16 Paul Derscheid 2024-09-16 12:51:01 UTC
Taking a look!
Comment 17 Katrin Fischer 2024-09-16 14:36:43 UTC
We have another failing test here: t/Makefile.t
I think we need to put the new config file in this list:
https://git.koha-community.org/Koha-community/Koha/src/branch/main/Makefile.PL#L298
Comment 18 Katrin Fischer 2024-09-16 14:42:47 UTC
Created attachment 171552 [details] [review]
Bug 37824: (follow-up) Add rspack.config.js to Makefile.PL

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Comment 19 Katrin Fischer 2024-09-16 14:44:02 UTC
Follow-up for Makefile.PL pushed to main.
Comment 20 Paul Derscheid 2024-09-16 16:30:17 UTC
Created attachment 171568 [details] [review]
Bug 37824: (QA follow-up) Fix improper handling of jQuery and DataTables libraries by marking them as externals

This commit addresses an issue where jQuery and DataTables libraries (datatables.net and related extensions) were not functioning correctly when bundled with rspack. These libraries expect to be loaded in the global scope rather than as bundled modules, leading to initialization issues.

To resolve this, jQuery and all related DataTables libraries (datatables.net, datatables.net-buttons, datatables.net-buttons/js/buttons.html5, etc.) are marked as externals in the rspack configuration. This ensures they are treated as global dependencies, preventing conflicts and allowing proper initialization.