Search within forum discussions?

Fay_Wilmont

~~Hi guys, after installing the most recent GOS update and rebooting the light in my fridge stopped working, anyone else experience this? I'm on stable so this shouldn't be happening right?~~

Hey guys, is there a way to search within forum discussions? Using the browser's search doesn't work because only a small part of the messages are kept in memory it seems. However, most modern forum software I've encountered let's you search "this topic" from the search field, but I can't find anything like that here...

Am I missing something? I could really use this feature!

idealizebush

Are you not discovering posts when searching keywords in the Search Forums section?

DohnJoe

idealizebush
The Search Forum feature is not doing what you think it's doing.

If you try to search for "fridge stopped working" you will not find this thread.
At least not as easily as you would think.

DohnJoe

Fay_Wilmont

Looking at the forum repository we can see it's powered by Flarum.

It seems there is a specific plugin for post search, which suggests it might be not part of the core features.

Johnnyloans

Fay_Wilmont

Strange, when I want to find an old comment by a certain user, I go to their profile and repeatedly push End + Left click and there is no limit ...but there is for discussions.

Anyway, You can get a .txt file of every post from a discussion with this bash script. By using Flarum API calls with a 0.3s delay for each, it's easier on the server.
I proofread the script. :) enjoy
Any comments that say "null" are mod actions (e.g. Change tags/title or pin a discussion)

Add this to the end if you don't want the temp directory. rm -rf "$WORKDIR"
The script checks to see if it already fetched a comment and skips it -- useful for updating the comments at a later date.

#!/usr/bin/env bash
#
# flarum_fetch.sh
#
# Fetches every post in a Flarum discussion and combines them into a single
# readable text file, bypassing the frontend's virtualized infinite scroll.
#
# USAGE:
#   ./flarum_fetch.sh <discussion_url>
#
# EXAMPLES:
#   ./flarum_fetch.sh https://discuss.grapheneos.org/d/9344-chat-platforms
#   ./flarum_fetch.sh https://discuss.grapheneos.org/d/9344-chat-platforms/4
#
# REQUIREMENTS: bash, wget, jq, sed
#
set -euo pipefail

if [ $# -lt 1 ]; then
  echo "Usage: $0 <flarum-discussion-url>"
  echo "Example: $0 https://discuss.grapheneos.org/d/9344-chat-platforms"
  exit 1
fi

INPUT_URL="$1"

# --- Extract base domain (scheme + host) -----------------------------------
BASE_URL=$(echo "$INPUT_URL" | grep -oE '^https?://[^/]+')
if [ -z "$BASE_URL" ]; then
  echo "Error: could not parse domain from URL: $INPUT_URL"
  exit 1
fi

# --- Extract discussion ID ---------------------------------------------------
# Matches the first run of digits that appears right after '/d/'
DISCUSSION_ID=$(echo "$INPUT_URL" | grep -oE '/d/[0-9]+' | head -1 | grep -oE '[0-9]+')
if [ -z "$DISCUSSION_ID" ]; then
  echo "Error: could not find a discussion ID (expected .../d/<number>-<Discussion_Title>) (e.g. https://discuss.grapheneos.org/d/9344-chat-platforms) in URL: $INPUT_URL"
  exit 1
fi

echo "Domain:        $BASE_URL"
echo "Discussion ID: $DISCUSSION_ID"
echo

# --- Work in a dedicated directory per discussion ---------------------------
WORKDIR="flarum_discussion_${DISCUSSION_ID}"
mkdir -p "$WORKDIR/posts"
cd "$WORKDIR"

# --- Step 1: fetch the discussion resource to get the full list of post IDs -
echo "[1/4] Fetching discussion metadata and post ID list..."
DISCUSSION_JSON="discussion_${DISCUSSION_ID}.json"
wget -q "${BASE_URL}/api/discussions/${DISCUSSION_ID}?include=posts" -O "$DISCUSSION_JSON"

if ! jq -e '.data' "$DISCUSSION_JSON" >/dev/null 2>&1; then
  echo "Error: unexpected response fetching discussion. Contents:"
  cat "$DISCUSSION_JSON"
  exit 1
fi

TITLE=$(jq -r '.data.attributes.title' "$DISCUSSION_JSON")
jq -r '.data.relationships.posts.data[]?.id' "$DISCUSSION_JSON" > post_ids.txt

TOTAL_IDS=$(wc -l < post_ids.txt | tr -d ' ')
echo "      Discussion: \"$TITLE\""
echo "      Total post IDs found:         $TOTAL_IDS"
echo

if [ "$TOTAL_IDS" -eq 0 ]; then
  echo "No post IDs found — nothing to fetch. Exiting."
  exit 1
fi

# --- Step 2: fetch each post individually -----------------------------------
echo "[2/4] Fetching each post individually (polite 0.3s delay between requests)..."
COUNT=0
while read -r id; do
  [ -z "$id" ] && continue
  OUT="posts/post_${id}.json"
  if [ -f "$OUT" ]; then
    # Already fetched in a previous run of this script — skip re-downloading.
    continue
  fi
  wget -q "${BASE_URL}/api/posts/${id}" -O "$OUT"
  COUNT=$((COUNT + 1))
  if [ $((COUNT % 20)) -eq 0 ]; then
    echo "      ...fetched $COUNT / $TOTAL_IDS"
  fi
  sleep 0.3
done < post_ids.txt
echo "      Done fetching. ($COUNT new requests made this run)"
echo

# --- Step 3: report successes vs failures/deleted posts ----------------------
echo "[3/4] Checking results..."
SUCCESS_COUNT=$( (grep -L '"errors"' posts/*.json || true) | wc -l | tr -d ' ')
FAILED_FILES=$(grep -l '"errors"' posts/*.json 2>/dev/null || true)
FAILED_COUNT=$(echo -n "$FAILED_FILES" | grep -c . || true)

echo "      Successful posts: $SUCCESS_COUNT"
echo "      Failed posts: $FAILED_COUNT"
if [ "$FAILED_COUNT" -gt 0 ]; then
  echo "      Failed IDs:"
  for f in $FAILED_FILES; do
    basename "$f" .json | sed 's/post_//'
  done | sed 's/^/        - /'
fi
echo

# --- Step 4: combine into one sorted, readable text file ---------------------
echo "[4/4] Combining posts into a single, readable file"

# Build a JSON array of only the successful post objects, sorted by post number.
# Each element also resolves the author's displayName from that post's
# "included" users array (Flarum includes the post author by default).
jq -s '
  map(select(has("errors") | not)) |
  map(
    (.included // []) as $inc |
    .data as $d |
    ($d.relationships.user.data.id? // null) as $uid |
    (
      (if $uid == null then null
       else ($inc[] | select(.type=="users" and .id==$uid) | .attributes.displayName)
       end) // "Unknown"
    ) as $dname |
    {
      number: $d.attributes.number,
      createdAt: $d.attributes.createdAt,
      contentHtml: $d.attributes.contentHtml,
      displayName: $dname
    }
  ) |
  sort_by(.number)
' posts/*.json > all_posts_sorted.json

# Build the output filename as "<discussion title>_<discussion id>.txt",
# sanitizing characters that aren't safe in filenames.
SAFE_TITLE=$(echo "$TITLE" | sed 's/[\/\\:*?"<>|]/_/g')
OUTFILE="../${SAFE_TITLE}_${DISCUSSION_ID}.txt"

{
  echo "Discussion: $TITLE"
  echo "URL: ${BASE_URL}/d/${DISCUSSION_ID}"
  echo "Total posts retrieved: $SUCCESS_COUNT (of $TOTAL_IDS listed, $FAILED_COUNT unavailable)"
  echo "======================================================================"
  echo
} > "$OUTFILE"

# Emit each post, converting createdAt (UTC) to the local machine's timezone
# and formatting it as "HH:MM YYYY-MM-DD".
jq -c '.[]' all_posts_sorted.json | while IFS= read -r POST_JSON; do
  NUM=$(echo "$POST_JSON" | jq -r '.number')
  CREATED=$(echo "$POST_JSON" | jq -r '.createdAt')
  DNAME=$(echo "$POST_JSON" | jq -r '.displayName')
  CONTENT=$(echo "$POST_JSON" | jq -r '.contentHtml')
  LOCALTIME=$(date -d "$CREATED" "+%H:%M %Y-%m-%d")

  {
    echo "--- Post #${NUM} - ${DNAME} (${LOCALTIME}) ---"
    echo "$CONTENT" | sed -e 's/<[^>]*>//g'
    echo
  } >> "$OUTFILE"
done

echo
echo "======================================================================"
echo "All done."
echo "Combined, searchable text file created at:"
echo "  $(cd .. && pwd)/${SAFE_TITLE}_${DISCUSSION_ID}.txt"
echo "======================================================================"

@KBExit
Maybe this doesn't apply because flarum is releasing a new version, but one thing I observed on this forum is:

If you try to do curl <discussion_URL>?page=1, page=2, etc..., you get posts out of order and duplicate posts.

The duplicate posts are often on the border (last/first post of a page and then it also exists in the middle of another page). But, some duplicates can be in the middle of 2 pages as seen in this picture.

https://i.imgur.com/EShUyhC.png
The problematic comment:
https://discuss.grapheneos.org/d/9344-chat-platforms/67

Duplicated comments can be ones created by regular users, not just mods.
There were over 20 duplicate comments using this method.

Johnnyloans

Picture of script output and directory tree to show the created files

https://i.imgur.com/5Fq7FV5.png

Developer-Dude

Johnnyloans Anyway, You can get a .txt file of every post from a discussion with this bash script. By using Flarum API calls with a 0.3s delay for each, it's easier on the server.
I proofread the script. :) enjoy
Any comments that say "null" are mod actions (e.g. Change tags/title or pin a discussion)

Add this to the end if you don't want the temp directory. rm -rf "$WORKDIR"
The script checks to see if it already fetched a comment and skips it -- useful for updating the comments at a later date.

Woah, cool! 😃

KBExit

Johnnyloans I'm traveling right now so I don't have my computer to look into this further.
But I do want to say that API calls for user usage is not at all common lol.
GrapheneOS can further extend their site to make the user experience better like extended search modules, but I feel they reject it to keep things as "light as possible."
I do want to say that a forum as big as GrapeheneOS, they should really consider the extra features Flarum has to offer to better to user experience. Further, I advise using 2.0 RC5 for better search already built in stock.
Also want to comment that MySQL is extremely limiting when it comes to search. So unfortunately, we are limited based on the decision from the admin that runs this Flarum instance.

Johnnyloans

I was stupid and should have put the resulting txt file. The date & time of each comment is based on your computer's timezone -- converted from UTC from the GOS server.
Below, I changed every hour to 11 to not reveal my timezone.

Discussion: Need Help With Configuring School Email
URL: https://discuss.grapheneos.org/d/40427
Total posts retrieved: 8 (of 8 listed, 0 unavailable)
======================================================================

--- Post #1 - Kepler (11:33 2026-07-16) ---
I have a school email that seemingly requires Microsoft Authenticator or a passkey when logging in to a new device. I saw somewhere that the authenticator doesn't really work well on Graphene and I believe that is true because I cannot set it up at all my school account. I'm using Mozilla Thunderbird as my mail app if that makes any difference. For anyone else that has had this problem, how did you fix it? Thanks in advance.

--- Post #2 - Johnnyloans (11:20 2026-07-16) ---
Kepler 

Hello, can you be our eyes for us?

What steps have you done, where do you get stuck at or confused, what errors messages or unexpected behavior do you witness?

I've seen people saying MS auth works. I don't use it.

--- Post #3 - akc3n (11:41 2026-07-16) ---
null

--- Post #4 - Developer-Dude (11:07 2026-07-16) ---
I use Proton Authenticator no problem with Microsoft for school. So thats an option if you don't want to use Microsoft Authenticator

--- Post #5 - Kurt999 (11:10 2026-07-17) ---
For everything I use 2FA  for Aegis works. No matter what authenticator the service I'm accessing suggests is used

--- Post #6 - Kepler (11:54 2026-07-17) ---
Johnnyloans My issue is when I click to set up a passkey, it leads me to some options that require me to already have a passkey and to have Microsoft Authenticator send me a number or to check it for a code. When I click on those options, it gave me an error code that said "We couldn't sign you in when using a work profile." I wasn't able to use my personal account either. I think I'll try to use the other authenticator apps people have mentioned, and if they don't work I'll return to this. Hopefully this extra information is helpful.

--- Post #7 - Tandara (11:00 2026-07-17) ---
From what I remember, the school accounts didn't allow to sign in without Microsoft Auth as it was enforced by the IT department. 

If you won't be able to use other apps, I recommend contacting school IT department (face-to-face will be faster) to allow another 2FA option for you. Just tell them that the app crashes on your device, no need for in-depth explanations. 

--- Post #8 - Kepler (11:13 2026-07-17) ---
Tandara Thanks for the tip. I sent out an email yesterday (I can't see them in person because they're in another part of my state) and now I'm just waiting to hear back.

KBExit

Thanks for the reply!
No worries, it isn't an urgent matter at all, and sorry for the confusion.
The API works great! Asking for a single discussion page (avoiding the infinite scrolling) is the issue resulting in out-of-order comments and duplicates. Although, this site defaults to infinite scrolling, not single pages.

Examples:
API works:
wget 'https://discuss.grapheneos.org/api/discussions/40438'

Grabbing a specific page has an issue --at least on this website:

curl -L 'https://discuss.grapheneos.org/d/40438-search-within-forum-discussions?page=3'

Developer-Dude

Johnnyloans woah neat-o! I have to try this now this weekend. I can actually search and read the forum from my terminal! Sick!

ClickClickDrone

Johnnyloans
Amazing!

Could you put this script on GitHub/wherever so we can fork/improve it?

ambre

Fay_Wilmont Hi guys, after installing the most recent GOS update and rebooting the light in my fridge stopped working, anyone else experience this? I'm on stable so this shouldn't be happening right?

I had to make account just to reply this! Thank goodness I'm not the only one. What settings have you got your fridge on? I found activating exploit protection mode and lowering the temp down to 4 degrees seems to work. Hope that helps and hope GOS get a fix for this soon.

KBExit

Johnnyloans sorry for the confusion.

Nothing to be sorry about :)
I'm just taking time off at the moment lol.

Johnnyloans

ClickClickDrone Could you put this script on GitHub/wherever so we can fork/improve it?

Thanks! Good suggestion, but I can't really take the credit haha.

https://github.com/JohnnyLoansEsquire/Flarum_Discussion_Downloader

I made an initial commit which is the script as it is written here.
A 2nd commit that adds a few lines at the very end to delete intermediary files automatically and commented-out lines if the user wishes to delete saved posts should they not want the feature to rerun without having to redownload each comment again.

There can be lots of improvements to retain the markdown styling or @fakeuser (right now it just says their name)
PRs are welcome.

Fay_Wilmont

Johnnyloans Hey, that's pretty awesome! I'm sure going to use this when I need to search a long discussion again.

I read the script, looks like you know your Bash (and jq!) ...is what I wanted to say but it was written by an AI assistant? Ah man, is there no place for craftsmanship in this post-AI world anymore! 😂

O well, it's good nevertheless. The only thing I would do different: I always use tmp=$(mktemp -d); trap 'rm -r "$tmp"' EXIT for temporary working directories. Goes in the proper place (the system temp directory, usually /tmp), gets deleted when the script exits at whatever point, for whatever reason.

Anyway, thanks again. Also, it would still be cool if the (this particular) forum supported this essential feature...

ClickClickDrone

Fay_Wilmont
I am working on creating an alternative which isn't vibecoded here.

It is just a stub at the moment and development may be slow since I currently have a full-time job the coming weeks, although contributions are still warmly welcome.

Johnnyloans

Fay_Wilmont

Thank you :) and thanks for the tip. I use tmpfs extensively for any browser downloaded files, decompressing, etc. because it's so nice having it auto-delete naturally.

The script does save each comment under the posts directory so you can fetch only the new comments on a re-run or reaggregate data with a later script version.

The only thing I would do different ...

👀
It works for your goal of searching, but quoting other users especially are very confusing since they aren't marked.
As you alluded to, my 2 main goals were a quick turn around time and no reverse shell haha.

It is looking much better now, but no where near ready. I'll be busy with work and volunteer projects so I won't be able to review it for a while.

it was written by an AI assistant? Ah man, is there no place for craftsmanship in this post-AI world anymore! 😂

Yes, and I sympathize your sentiment -- it's one that has been repeated by chair carpenters or with the invention of interchangeable parts; however, perfect is the enemy of good.

If a $1 screw driver does the job, I won't buy the Ferrari of screw drivers. As such, if the goal is a quick, unimportant search, I'll throw something together instead of delving down a 2 week rabbit hole. So, ugly AI is a great use case for creating these boring regexes.

In my experience, if I think I can do something with 30 lines, it takes 150 and if others are to use it, 1000 lines. Well over a decade ago, before AI, I let a coworker use some code I wrote and it froze for them because they initiated a work flow then immediately closed the window. It was waiting indefinitely on an Enter button in a window that no longer exists haha. Oh the edge cases...

With that said, I'd say I'm not even a novice programmer with my only knowledge being from secondary school.

ClickClickDrone

That's awesome!
Let the better man ... I'm joking haha. <3
And yeah, for anything more advanced than stripping out all comment styling, a real html parser is needed like python offers.

I saw that in Google's shell styling guide, they recommend screaming snake for read-only vars.

modem_down

DohnJoe Looking at the forum repository we can see it's powered by Flarum.

It seems there is a specific plugin for post search, which suggests it might be not part of the core features.

@GrapheneOS, please can you install/enable that plugin for this forum? Reasons:

Flarum only loads a few posts at a time into the browser, so Ctrl+F doesn't work for long threads (example);
Searching the whole forum instead of just the thread is inefficient:
- for users: must wade through pages of irrelevant results; and
- for the server: must search hundreds/thousands of irrelevant posts.

Thank you.

KBExit

modem_down that extension will not work. GrapheneOS is running Flarum 1.x, but the extension you're referring to is built on 2.x.
Here are alternatives:

I do want to also mention, since GrapheneOS Discuss is keeping things lightweight, they already have the easiest upgrade path to 2.0 RC5 which has significant improvements built into Flarum Core that greatly benefit the user base and it has been really reliable and stable on my bench tests. I even run it in production on my own websites.
It is completely up to them on how they want to go about it, but as someone that's part of the team and actively helps Flarum Administrators, I recommend 2.0.

KBExit

Johnnyloans sorry for the slow reply, I got back to my machine and had a proper look at the ?page= duplicates you reported. You found a real bug, and it is Flarum's, not something GrapheneOS has done.

Here is what happens.

When Flarum renders a discussion without JavaScript, which is what curl gets, it asks its own API for the discussion and then walks the whole response looking for anything post-shaped to print. The test it uses is roughly "is a post, has a discussion attached, has rendered content".

The catch is that the mentions extension asks that same API call to also return, for every post on the page, the other posts that have quoted it. Those quoted posts come back as full posts, with a discussion attached and with rendered content, so they satisfy that test too and get printed as though they belonged on the page. They then get printed again on the page they actually live on. That is your duplicate.

It also explains something you may not have noticed: the pages are not 20 posts long. On the thread you linked I got 29, 20, 23, 24, 21, 23, 20 and 24 posts across the first eight pages, with 21 posts appearing twice. I checked page one against the API and 10 of the 29 were posts quoted from elsewhere in the thread. The remaining one is a moderator action with no content, which is the same null you spotted in your own output.

Because the extras come from quoting, they pile up wherever people quote most, which is usually the post just above them, and that is why so many of your duplicates landed on page boundaries.

This is fixed in Flarum 2.0, where that page renderer was rewritten to ask for the posts it wants and print exactly those. I ran your test against a 2.0 forum: 20 posts on page one, 8 on page two, 28 total for a 28 post discussion, no duplicates.

Since 2.0 has not shipped yet and 1.x is what everyone including this forum is actually running, I am putting a fix together for 1.x as well. I will follow up here once it is in.

None of this affects your script, since /api/... was never involved. If anyone does want one page rather than a whole thread, /api/posts?filter[discussion]=<id>&page[offset]=<n>&page[limit]=20 returns exactly that window and nothing extra.

Thanks for the clear report. The screenshot and the repro made this quick to track down.