Bug 34315 - Provide an alternative to the mailing lists - Discourse
Summary: Provide an alternative to the mailing lists - Discourse
Status: NEW
Alias: None
Product: Project Infrastructure
Classification: Unclassified
Component: Project website (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Liz Rea
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-07-19 22:12 UTC by Pablo AB
Modified: 2023-12-23 20:48 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Pablo AB 2023-07-19 22:12:26 UTC
Back in 2021 Jonathan Druart mention the option to move out from Mailman to Flarum or Discourse:

https://www.mail-archive.com/koha-devel@lists.koha-community.org/msg12359.html
Comment 1 David Cook 2023-12-06 23:06:54 UTC
I currently run a Discourse instance so happy to share what I know...
Comment 2 Thomas Dukleth 2023-12-13 13:32:17 UTC
Do you propose to fix the won't fix status of search by date?
Comment 3 Thomas Dukleth 2023-12-13 13:45:59 UTC
Consideration should be given to Hyperkitty, the Mailman 3 forum software.  This allows having forum software with proper mailing list support and certainly searching by date which is a basic feature does work in Hyperkitty.
Comment 4 David Cook 2023-12-13 22:54:51 UTC
(In reply to Thomas Dukleth from comment #2)
> Do you propose to fix the won't fix status of search by date?

Who are you talking to here?
Comment 5 David Cook 2023-12-13 22:57:05 UTC
(In reply to Thomas Dukleth from comment #3)
> Consideration should be given to Hyperkitty, the Mailman 3 forum software. 
> This allows having forum software with proper mailing list support and
> certainly searching by date which is a basic feature does work in Hyperkitty.

I'd never heard of Hyperkitty, but it looks like stunnel uses it: https://www.stunnel.org/mailman3/hyperkitty/

No idea what you mean by "searching by date" in this context. Are you referring to browse rather than sort? Like hyperkitty's thread archive? https://www.stunnel.org/mailman3/hyperkitty/list/stunnel-announce@stunnel.org/
Comment 6 David Cook 2023-12-13 23:01:07 UTC
I don't think Discourse has something like that but you can search using before/after date filters.
Comment 7 Thomas Dukleth 2023-12-15 17:49:03 UTC
(In reply to David Cook from comment #4)
> (In reply to Thomas Dukleth from comment #2)
> > Do you propose to fix the won't fix status of search by date?
> 
> Who are you talking to here?

By "who" I had meant us.  I thought I had found some months ago that the basic feature of searching by date was missing from Discourse and that the developers had some problem about creating a search by date feature which I found truly shocking.

I should have corrected my mistake more quickly in this bug once I had actually posted incorrectly here.  I quickly discovered on my own after Wednesday's development meeting that I had been mistaken about Discourse missing a search by date feature and corrected myself on IRC.

A few months ago I had examined Discourse in some detail and I had been shocked to mistakenly discover that it did not have a search by date feature because the friendly advanced search user interface was too well hidden behind a hierarchy of skeuomorphic icons.  Additionally, I remembered having found some report about an enhancement request relating to time period which had been designated won't fix in some manner.  While it seemed difficult to believe that such a basic feature seemed to be missing, the tone which people from the company which controls Discourse use on their site sometimes seems inappropriately harsh when they are uninterested in some line of development.  These combination of factors  had led me to believe that searching by date did not exist in Discourse and was in some way difficult to implement for which I had imagined that the internal date representation for posts was confused by multiple dates and possibly not always preserved in a manner which people would find most helpful.

Most likely the request about date searching which I remember finding months ago but did not find again now related to adding a user interface feature for searching a time period between two dates which mistook for searching for dates in any way.  The before and after syntax is available, but I understand now that adding the combination of before and after to the advanced search for time period might be confusing to some users.

I remember when a link to a properly labelled advanced search would be commonly found next to a simple search box.  Some popular sites may have found that an advanced search feature while highly valued by some users would scare more people away on some popular websites.  Consequently, it has become fashionable to hide advanced search.  It has also become common to hide even simple search with a magnifying glass icon.  Discourse is the first program for which I have now noticed that advanced search is hidden behind another menu icon after first clicking the magnifying glass icon.
Comment 8 Thomas Dukleth 2023-12-15 18:22:57 UTC
(In reply to David Cook from comment #5)
> (In reply to Thomas Dukleth from comment #3)
> > Consideration should be given to Hyperkitty, the Mailman 3 forum software. 
> > This allows having forum software with proper mailing list support and
> > certainly searching by date which is a basic feature does work in Hyperkitty.

Correcting myself now, searching by date appears to work much better in Discourse than in HyperKitty.

> 
> I'd never heard of Hyperkitty, but it looks like stunnel uses it:
> https://www.stunnel.org/mailman3/hyperkitty/
> 
> No idea what you mean by "searching by date" in this context. Are you
> referring to browse rather than sort? Like hyperkitty's thread archive?
> https://www.stunnel.org/mailman3/hyperkitty/list/stunnel-announce@stunnel.
> org/

Browsing by date is an obvious feature of the side navigation column in HyperKitty which may have the advantage of being able to go the next point in the date sequence much more efficiently.

Searching by date is a much more accurate and flexible in Discourse but is less efficient for browsing messages through the sequence of all messages matching a date criteria.  Searching by date could be fixed for HyperKitty using Django upon which HyperKitty runs.

Using multiple web browser tabs, to link from the result set in one tab is needed in Discourse to preserve the search results and link to the next post in a new tab.  If there is additional text in the query in addition to date multiple tabs are needed in both Django and HyperKitty.  Very old mail clients do this better but I expect it can be fixed in both.
Comment 9 Thomas Dukleth 2023-12-21 11:02:28 UTC
I have made a much more detailed investigation of many aspects of Mailman 3 with Postorius and HyperKitty, Discourse, and Flarum.  I will post details as I consolidate text for the various aspects.

We need to remain inclusive and retain email compatibility for the portion of mailing list subscribers is who rightly understand that forum software is often designed to make staying informed about about several threads an inefficient game.  Community development may be aided by having some gamification available even if it is accompanied by anti-usability features.


Flarum Misses Email Compatibility.

Flarum should be excluded from consideration at least on the grounds of breaking inclusiveness for having the feature for users to send email only as a recently under development probably only proprietary and depending upon a proprietary service.


HyperKitty Misses Some Additional Forum Development.

HyperKitty may be a weak choice for the forum aspect because it lacks development effort past an early point.  It has only the most basic features and might not need much more other than hooking into some additional Django features and plugins along with a more engaging CSS but the absence of that work leaves HyperKitty less likely to assist as much as Discourse with community engagement for which the anti-usability features of Discourse may be a virtue.  I will detail what is committed but not released and the problem of Fedora development of HyperKitty which is divergent and for which changes are not committed upstream.  [The Fedora development does not have much more of what is needed but does have some commits which could be adopted upstream.] 

[Search by date in HyperKitty, unlike browse by date which obviously works, seems to be inaccurate for lack of good date indexing but may work properly with appropriate undocumented syntax.  Sorting search results by date is easy and the web browser back function always works to return to the place in the search result set in my more recent testing of HyperKitty with no need for multiple browser tabs.  HyperKitty is still much less efficient than very old email clients.] 


Discourse Manageable with Configuration and Modification Contrary to Design. 

Discourse seems the best choice for forum engagement despite some terrible inefficient anti-usability features merely by the absence of a competing level FOSS development work for some other forum.  Effective configuration for a friendly cooperative and inclusive FOSS community requires much configuration counter to the intentions of Civilized Discourse Construction Kit, the company controlling Discourse with the use of a contributors license agreement and much else.  Good FOSS options should also be chosen contrary to the official recommendation of proprietary services which may be easier for most users to implement.  I will detail all those which I discovered after consolidating my text for various aspects.

In the case of receiving email. what is officially described as the worst choice should be considered the best.  Living up to the promise of email compatibility will require either modifying a hard coded design, creating some plugin to do so, or constantly running a script to elevate everyone to trust level 2 merely to allow known users to use email for posting with their own topic as opposed to merely answering about topics raised by others without being forced to use the forum software for an extended period.  Email compatibility for those who rightly find forum software especially problematic should mean not forcing users to use the forum software.

[A search result set can be forced to a state where the web browser back function at least partially works by following the more link with less need for multiple browser tabs to preserve context.  It is still much less efficient than very old email clients.]
Comment 10 Thomas Dukleth 2023-12-23 20:48:08 UTC
[Correcting myself.]

There is a Discourse option for setting the minimum trust level of users for creating a topic by email.  Unfortunately, the option is ambiguously named and easy to mistake for a more general function of having Discourse receive email.  Yet, the option exists and is easy to reduce from default trust level 2 to trust level 0 matching replying by email to an existing topic and allowing people who work more efficiently using email to do so without unnecessary extended use of inefficient forum software.

[There is much hardcoded behaviour from the developers at Civilized Discourse Construction Set, the company which controls Discourse, who often do not understand why anyone would prefer otherwise but creating new topics by email does have an option for setting the minimum trust level instead of the default trust level 2 value.  Lack of formal documentation beyond the most basic level for installation is problematic.  Most of what passes for documentation is either the code [upon which I have needed to rely too much]; the user interface; or very brief explanations of some narrow subtopic often in answer to a specific question buried in https://meta.discourse.org .]

The ambiguously named options for email_in* are distinct from reply_by_email* despite "reply by email" needing to be received as "email in" the Discourse instance in order to function.  Perhaps the ambiguity in naming is an artefact of historical development order.

Replying by email depends upon a token which incorporates both the topic and the user sending the reply.

Creating a new topic by sending an email message depends on the special email address for the category in which the topic [email subject] would go.  Categories might be very broad such as devel, translate, etc. and use tags for faceted discovery.  There is no provision for assigning tags to a topic in Discourse from sending an email message but tags beyond list name are generally not a feature of exchanging mailing list email even if that could be useful.  [Using the forum software for tag assignment also helps avoid proliferation of tag names from misspellings and unnecessary different names if users avoid such with autocompletion in tag assignment.]  

email_in:
(Allow users to post new topics via email.  After enabling this setting, you will be able to configure incoming email addresses for groups and categories.)

email_in_min_trust:
(The minimum trust level a user needs to have to be allowed to post new topics via email.)

reply_by_email_enabled:
(Enable replying to topics via email)

To my knowledge, there is no reply_by_email_min_trust option.  Replying to an existing topic is considered to have less need for trust in Discourse which is either enabled for all users of for none and seems to be hard coded at trust level 0 if enabled which is fine.  The trust level for posting a new topic is considered more concerning for spam abuse.

There are better ways of controlling spam than restricting users from doing most of what people need to do for communicating regularly on mailing lists.  The developers at Civilized Discourse Construction Kit, approach spam control by having very little restriction for spammers to create an account but excessive restrictions for anyone to contribute usefully.  Reducing the likelihood that spammers would have be approved to obtain an account is better than deleting spam which could have been avoided.  Spam in topic creation is likely to be much more obvious and thus stopped quickly and especially when created for email can be reduced by email spam checks more easily than topic reply spam because the email subject header is used for topic creation.  Spam control for email before it would even be available to Discourse can make email submission the least likely to be the source to Discourse post spam but enabling email spam control requires configuration for incoming mail the opposite of recommendation in Discourse.

There are other defaults which need to be changed for email compatibility in Discourse to be meaningful without being forced to use the forum software for an extended period.  Here, I have merely corrected my error about the presence of an option and identified the difficulty of default choices to which many Discourse users have objected.  There are also problematic hard coded issues which we do not have for email_in_min_trust. 

(In reply to Thomas Dukleth from comment #9)

[...]

> Living up to the promise of email
> compatibility will require either modifying a hard coded design, creating
> some plugin to do so, or constantly running a script to elevate everyone to
> trust level 2 merely to allow known users to use email for posting with
> their own topic as opposed to merely answering about topics raised by others
> without being forced to use the forum software for an extended period.