This is a massive PR, so apologies for the poor performance viewing it!
This merges in the content of Mastodon 4.3.0. *This PR isn't ready to
deploy to production*, and I don't recommend trying it in a production
environment. Since I started by merging 4.3.0, this will likely have
reverted at least some of the content of the post-4.3.0 security patches
in the 4.2.x series; those will be brought back as I merge in 4.4.0 and
4.5.0.
I haven't yet tested this PR, so it may not actually run without
additional tweaks. I'm putting it up now for discussion and so multiple
people can test if we want.
---
Here's a few notes on things I ran into and thoughts I had while working
on it:
* We may want to look at `server_banner.jsx,` where I ended up undoing
the v3-style rewrite due to the new react component being used. I'm not
sure what the current goal for it is.
* I may not have 100% merged in all content from `status_action_bar`
correctly.
* There's a number of places where I updated the links for remote
usernames, and they should be tested to make sure they work. They're all
`<Permalink>` elements. I have in my notes that `status_content` might
need double-checking.
* I may also not have gotten the spoiler display right in
`status_content`.
* It looks like previous versions of Hometown had customizations to the
`media_item` display, not all of which was preserved as-is due to
surrounding rewrites. I'm not sure what the vision is and could use
another set of eyes.
* The character count rendering clashed with the newly-introduced
handling of a character count variable upstream, and I'm not sure if the
version I've got here 100% works or not.
* The navigation bar was completely rewritten and no longer has the
components Hometown was customizing; did I properly catch any
replacements? Did we handle all links to accounts to make sure they go
to the remote instance?
* Polls: I reverted the single/multiple choice toggle because upstream
has an actual proper UI for this that should be integrated in one of the
versions we’re merging in. Please double-check that the current version
looks fine to you.
* The font icons will need double-checking. Hometown made some custom
use of the font-awesome icons, while Mastodon 4.3.0 replaced
font-awesome with something else. I may not have caught all places that
font-awesome icons were being used.
* We’ll want to make sure the post display is correct, including
Hometown customizations, because the previous static post page has been
replaced with the React one.
---------
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Co-authored-by: Matt Jankowski <matt@jankowski.online>
Co-authored-by: Renaud Chaput <renchap@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: David Roetzel <david@roetzel.de>
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
Co-authored-by: Jeong Arm <kjwonmail@gmail.com>
Co-authored-by: Christian Schmidt <github@chsc.dk>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <noreply@github.com>
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Darius Kazemi <darius.kazemi@gmail.com>
Co-authored-by: diondiondion <mail@diondiondion.com>
Co-authored-by: Echo <ChaosExAnima@users.noreply.github.com>
Co-authored-by: Shugo Maeda <shugo.maeda@gmail.com>
Co-authored-by: emilweth <7402764+emilweth@users.noreply.github.com>
Co-authored-by: Shlee <github@shl.ee>
Co-authored-by: Joshua Rogers <MegaManSec@users.noreply.github.com>
Co-authored-by: Jessica Stokes <hello@jessicastokes.net>
Co-authored-by: PGray <77597544+PGrayCS@users.noreply.github.com>
* fix(status): remove send usage for private unlink_from_conversations
- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service
* fix(feeds_vacuum): replace find_in_batches with in_batches
because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
* refactor(statuses_vacuum): remove dead code - unused
Method is not called inside class and private.
Clean up dead code.
* refactor(statuses_vacuum): make retention_period present test explicit
This private method only hides functionality.
It is best practice to be as explicit as possible.
* refactor(statuses_vacuum): improve query performance
- fix statuses_scope having sub-select for Account.remote scope by
`joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
because it is inexplicit, bad practice and even slower than normal
`.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
because in_batches delivers a full blown active record query result,
in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send
* refactor(status): simplify unlink_from_conversations
- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?
* refactor(status): searchable_by way less sub selects
These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.
This is how it was:
```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
Status Load (1.6ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]]
Mention Load (1.5ms) SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
QUERY PLAN
------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.15..23.08 rows=1 width=41)
-> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8)
Filter: (domain IS NULL)
-> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41)
Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
Filter: (NOT silent)
(6 rows)
```
This is how it is with this change:
```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
Status Load (1.7ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]]
Mention Load (0.7ms) SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
QUERY PLAN
------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.15..23.08 rows=1 width=41)
-> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8)
Filter: (domain IS NULL)
-> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41)
Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
Filter: (NOT silent)
(6 rows)
```
Misty De Méo
Matt Jankowski
Emelia Smith
jsgoldstein
Claire
Eugen Rochko
Nick Schonning
Kaspar V