This is a massive PR, so apologies for the poor performance viewing it!
This merges in the content of Mastodon 4.3.0. *This PR isn't ready to
deploy to production*, and I don't recommend trying it in a production
environment. Since I started by merging 4.3.0, this will likely have
reverted at least some of the content of the post-4.3.0 security patches
in the 4.2.x series; those will be brought back as I merge in 4.4.0 and
4.5.0.
I haven't yet tested this PR, so it may not actually run without
additional tweaks. I'm putting it up now for discussion and so multiple
people can test if we want.
---
Here's a few notes on things I ran into and thoughts I had while working
on it:
* We may want to look at `server_banner.jsx,` where I ended up undoing
the v3-style rewrite due to the new react component being used. I'm not
sure what the current goal for it is.
* I may not have 100% merged in all content from `status_action_bar`
correctly.
* There's a number of places where I updated the links for remote
usernames, and they should be tested to make sure they work. They're all
`<Permalink>` elements. I have in my notes that `status_content` might
need double-checking.
* I may also not have gotten the spoiler display right in
`status_content`.
* It looks like previous versions of Hometown had customizations to the
`media_item` display, not all of which was preserved as-is due to
surrounding rewrites. I'm not sure what the vision is and could use
another set of eyes.
* The character count rendering clashed with the newly-introduced
handling of a character count variable upstream, and I'm not sure if the
version I've got here 100% works or not.
* The navigation bar was completely rewritten and no longer has the
components Hometown was customizing; did I properly catch any
replacements? Did we handle all links to accounts to make sure they go
to the remote instance?
* Polls: I reverted the single/multiple choice toggle because upstream
has an actual proper UI for this that should be integrated in one of the
versions we’re merging in. Please double-check that the current version
looks fine to you.
* The font icons will need double-checking. Hometown made some custom
use of the font-awesome icons, while Mastodon 4.3.0 replaced
font-awesome with something else. I may not have caught all places that
font-awesome icons were being used.
* We’ll want to make sure the post display is correct, including
Hometown customizations, because the previous static post page has been
replaced with the React one.
---------
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Co-authored-by: Matt Jankowski <matt@jankowski.online>
Co-authored-by: Renaud Chaput <renchap@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: David Roetzel <david@roetzel.de>
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
Co-authored-by: Jeong Arm <kjwonmail@gmail.com>
Co-authored-by: Christian Schmidt <github@chsc.dk>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <noreply@github.com>
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Darius Kazemi <darius.kazemi@gmail.com>
Co-authored-by: diondiondion <mail@diondiondion.com>
Co-authored-by: Echo <ChaosExAnima@users.noreply.github.com>
Co-authored-by: Shugo Maeda <shugo.maeda@gmail.com>
Co-authored-by: emilweth <7402764+emilweth@users.noreply.github.com>
Co-authored-by: Shlee <github@shl.ee>
Co-authored-by: Joshua Rogers <MegaManSec@users.noreply.github.com>
Co-authored-by: Jessica Stokes <hello@jessicastokes.net>
Co-authored-by: PGray <77597544+PGrayCS@users.noreply.github.com>
- Align webfinger_controller with upstream
- Remove validation from webhook (It's not in v4.2.1, I don't know where it came from)
- Remove show_application from other view (merge error)
- Remove duplicate display name from account header
- Fix misspelled className for navigation bar
It looks like a [bug](https://bugs.ruby-lang.org/issues/18633) around
autosplat is [fixed](fbaadd1cfe)
on ruby-3.2.0-rc1 and breaks a test (but not on ruby <= 3.1.3):
```
$ bundle exec rspec ./spec/controllers/api/v1/emails/confirmations_controller_spec.rb:41
:
1) Api::V1::Emails::ConfirmationsController#create with an oauth token from an app that created the account when the account is already confirmed but user changed e-mail and has not confirmed it returns http success
Failure/Error:
def email_changed(user, **)
@resource = user
@instance = Rails.configuration.x.local_domain
return unless @resource.active_for_authentication?
I18n.with_locale(locale) do
mail to: @resource.email, subject: I18n.t('devise.mailer.email_changed.subject')
end
end
ArgumentError:
wrong number of arguments (given 2, expected 1)
# ./app/mailers/user_mailer.rb:51:in `email_changed'
# ./app/models/user.rb:444:in `render_and_send_devise_message'
# ./app/models/user.rb:430:in `block in send_pending_devise_notifications'
# ./app/models/user.rb:429:in `each'
# ./app/models/user.rb:429:in `send_pending_devise_notifications'
# ./spec/controllers/api/v1/emails/confirmations_controller_spec.rb:38:in `block (7 levels) in <top (required)>'
```
* Add trending statuses
* Fix dangling items with stale scores in localized sets
* Various fixes and improvements
- Change approve_all/reject_all to approve_accounts/reject_accounts
- Change Trends::Query methods to not mutate the original query
- Change Trends::Query#skip to offset
- Change follow recommendations to be refreshed in a transaction
* Add tests for trending statuses filtering behaviour
* Fix not applying filtering scope in controller
* Add appeals
* Add ability to reject appeals and ability to browse pending appeals in admin UI
* Add strikes to account page in settings
* Various fixes and improvements
- Add separate notification setting for appeals, separate from reports
- Fix style of links in report/strike header
- Change approving an appeal to not restore statuses (due to federation complexities)
- Change style of successfully appealed strikes on account settings page
- Change account settings page to only show unappealed or recently appealed strikes
* Change appealed_at to overruled_at
* Fix missing method error
* Add trending links
* Add overriding specific links trendability
* Add link type to preview cards and only trend articles
Change trends review notifications from being sent every 5 minutes to being sent every 2 hours
Change threshold from 5 unique accounts to 15 unique accounts
* Fix tests
* Fix misuse of foreign_type
* Fix use of removed "add_template_helper"
* Use response.media_type instead of response.content_type in tests
* Fix CSV export controller test on Rails 6
Rails 6 sets a "filename*" field in the Content-Disposition header to
explicitly encode the filename as UTF-8.
This changes checks the first part of the Content-Disposition header so
it matches in both Rails 5 and Rails 6.
* Fix emoji formatting with Rails 6
* Make emoji output more idiomatic and robust
* Switch from redis-rails gem to built-in Rails redis cache storage
* feat: add possibility of adding WebAuthn security keys to use as 2FA
This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: add request for WebAuthn as second factor at login if enabled
This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: add possibility of deleting WebAuthn Credentials
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: disable WebAuthn when an Admin disables 2FA for a user
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA
Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.
* We had to change a little the flow for setting up TOTP, given that now
it's possible to setting up again if you already had TOTP, in order to
let users modify their authenticator app – given that now it's not
possible for them to disable TOTP and set it up again with another
authenticator app.
So, basically, now instead of storing the new `otp_secret` in the
user, we store it in the session until the process of set up is
finished.
This was because, as it was before, when users clicked on 'Edit' in
the new two-factor methods lists page, but then went back without
finishing the flow, their `otp_secret` had been changed therefore
invalidating their previous authenticator app, making them unable to
log in again using TOTP.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* refactor: fix eslint errors
The PR build was failing given that linting returning some errors.
This commit attempts to fix them.
* refactor: normalize i18n translations
The build was failing given that i18n translations files were not
normalized.
This commits fixes that.
* refactor: avoid having the webauthn gem locked to a specific version
* refactor: use symbols for routes without '/'
* refactor: avoid sending webauthn disabled email when 2FA is disabled
When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.
* refactor: avoid creating new env variable for webauthn_origin config
* refactor: improve flash error messages for webauthn pages
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
Fix#271
Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.
When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.
The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.
The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
- Reduce time-to-digest from 20 to 7 days
- Fetch mentions starting from +1 day since last login
- Fix case when last login is more recent than last e-mail
- Do not render all mentions, only 40, but show number in subject
- Do not send digest to moved accounts
- Do send digest to silenced accounts
* Add moderation warnings
Replace individual routes for disabling, silencing, and suspending
a user, as well as the report update route, with a unified account
action controller that allows you to select an action (none,
disable, silence, suspend) as well as whether it should generate an
e-mail notification with optional custom text. That notification,
with the optional custom text, is saved as a warning.
Additionally, there are warning presets you can configure to save
time when performing the above.
* Use Account#local_username_and_domain
* Fix#201: Account archive download
* Export actor and private key in the archive
* Optimize BackupService
- Add conversation to cached associations of status, because
somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
(Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
(Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
(Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
GIF-to-MP4 media are all named "media.mp4" for some reason
* Keep uniquely generated filename in Paperclip::GifTranscoder
* Ensure dumped files do not overwrite each other by maintaing directory partitions
* Give tar archives a good name
* Add scheduler to remove week-old backups
* Fix code style issue
Misty De Méo
Claire
nachtjasmin
Christian Schmidt
Matt Jankowski
Eugen Rochko
Heitor de Melo Cardozo
Nick Schonning
zunda
Jeong Arm
helloworldstack
santiagorodriguez96
Renato "Lond" Cerqueira
Takeshi Umeda
Emma Winston
ysksn
Quint Guvernator
abcang