Browse Source
* peepeepoopoo * test domain perm subs * swagger * envparsing * dries your wets * start on docs * finish up docs * copy paste errors * rename actions package * rename force -> skipCache * move obfuscate parse nearer to where err is checked * make higherPrios a simple slice * don't use receiver for permsFrom funcs * add more context to error logs * defer finished log * use switch for permType instead of if/else * thanks linter, love you <3 * validate csv headers before full read * use bufio scannerpull/3639/head
tobi
1 year ago
committed by
GitHub
95 changed files with 3320 additions and 626 deletions
@ -0,0 +1,145 @@
|
||||
# Domain Permission Subscriptions |
||||
|
||||
Via the [admin settings panel](./settings.md#subscriptions), you can create and manage domain permission subscriptions. |
||||
|
||||
Domain permission subscriptions allow you to specify a URL at which a permission list is hosted. Every 24hrs at 11pm (by default), your instance will fetch and parse each list you're subscribed to, in order of priority (highest to lowest), and create domain permissions (or domain permission drafts) based on entries discovered in the lists. |
||||
|
||||
Each domain permission subscription can be used to create domain allow or domain block entries. |
||||
|
||||
!!! warning |
||||
Currently, via blocklist subscriptions it is only possible to create "suspend" level domain blocks; other severities are not yet supported. Entries of severity "silence" or "limit" etc. on subscribed blocklists will be skipped. |
||||
|
||||
## Priority |
||||
|
||||
When you specify multiple domain permission subscriptions, they will be fetched and parsed in order of priority, from highest priority (255) to lowest priority (0). |
||||
|
||||
Permissions discovered on lists higher up in the priority ranking will override permissions on lists lower down in the priority ranking. |
||||
|
||||
For example, an instance admin subscribes to two allow lists, "Important List" at priority 255, and "Less Important List" at priority 128. Each of these subscribed lists contain an entry for `good-eggs.example.org`. |
||||
|
||||
The subscription with the higher priority is the one that now creates and manages the domain allow entry for `good-eggs.example.org`. |
||||
|
||||
If the subscription with the higher priority is removed, then the next time all the subscriptions are fetched, "Less Important List" will create (or take ownership of) the domain allow instead. |
||||
|
||||
## Orphan Permissions |
||||
|
||||
Domain permissions (blocks or allows) that are not currently managed by a domain permission subscription are considered "orphan" permissions. This includes permissions that an admin created in the settings panel by hand, or which were imported manually via the import/export page. |
||||
|
||||
If you wish, when creating a domain permission subscription, you can set ["adopt orphans"](./settings.md#adopt-orphan-permissions) to true for that subscription. If a domain permission subscription that is set to adopt orphans encounters an orphan permission which is *also present on the list at the subscription's URI*, then it will "adopt" the orphan by setting the orphan's subscription ID to its own ID. |
||||
|
||||
For example, an instance admin manually creates a domain block for the domain `horrid-trolls.example.org`. Later, they create a domain permission subscription for a block list that contains an entry for `horrid-trolls.example.org`, and they set "adopt orphans" to true. When their instance fetches and parses the list, and creates domain permission entries from it, then the orphan domain block for `horrid-trolls.example.org` gets adopted by the domain permission subscription. Now, if the domain permission subscription is removed, and the option to remove all permissions owned by the subscription is checked, then the domain block for `horrid-trolls.example.org` will also be removed. |
||||
|
||||
## Fun Stuff To Do With Domain Permission Subscriptions |
||||
|
||||
### 1. Create an allowlist-federation cluster. |
||||
|
||||
Domain permission subscriptions make it possible to easily create allowlist-federation clusters, ie., a group of instances can essentially form their own mini-fediverse, wherein each instance runs in [allowlist federation mode](./federation_modes.md#allowlist-federation-mode), and subscribes to a cooperatively-managed allowlist hosted somewhere. |
||||
|
||||
For example, instances `instance-a.example.org`, `instance-b.example.org`, and `instance-c.example.org` decide that they only want to federate with each other. |
||||
|
||||
Using some version management platform like GitHub, they host a plaintext-formatted allowlist at something like `https://raw.githubusercontent.com/our-cluster/allowlist/refs/heads/main/allows.txt`. |
||||
|
||||
The contents of the plaintext-formatted allowlist are as follows: |
||||
|
||||
```text |
||||
instance-a.example.org |
||||
instance-b.example.org |
||||
instance-c.example.org |
||||
``` |
||||
|
||||
Each instance admin sets their federation mode to `allowlist`, and creates a subscription to create allows from `https://raw.githubusercontent.com/our-cluster/allowlist/refs/heads/main/allows.txt`, which results in domain allow entries being created for their own domain, and for each other domain in the cluster. |
||||
|
||||
At some point, someone from `instance-d.example.org` asks (out of band) whether they can be added to the cluster. The existing admins agree, and update their plaintext-formatted allowlist to read: |
||||
|
||||
```text |
||||
instance-a.example.org |
||||
instance-b.example.org |
||||
instance-c.example.org |
||||
instance-d.example.org |
||||
``` |
||||
|
||||
The next time each instance fetches the list, a new domain allow entry will be created for `instance-d.example.org`, and it will be able to federate with the other domains on the list. |
||||
|
||||
### 2. Cooperatively manage a blocklist. |
||||
|
||||
Domain permission subscriptions make it easy to collaborate on and subscribe to shared blocklists of domains that host illegal / fashy / otherwise undesired accounts and content. |
||||
|
||||
For example, the admins of instances `instance-e.example.org`, `instance-f.example.org`, and `instance-g.example.org` decide that they are tired of duplicating work by playing whack-a-mole with bad actors. To make their lives easier, they decide to collaborate on a shared blocklist. |
||||
|
||||
Using some version management platform like GitHub, they host a blocklist at something like `https://raw.githubusercontent.com/baddies/blocklist/refs/heads/main/blocks.csv`. |
||||
|
||||
When someone discovers a new domain hosting an instance they don't like, they can open a pull request or similar against the list, to add the questionable instance to the domain. |
||||
|
||||
For example, someone gets an unpleasant reply from a new instance `fashy-arseholes.example.org`. Using their collaboration tools, they propose adding `fashy-arseholes.example.org` to the blocklist. After some deliberation and discussion, the domain is added to the list. |
||||
|
||||
The next time each of `instance-e.example.org`, `instance-f.example.org`, and `instance-g.example.org` fetch the block list, a block entry will be created for ``fashy-arseholes.example.org``. |
||||
|
||||
### 3. Subscribe to a blocklist, but ignore some of it. |
||||
|
||||
Say that `instance-g.example.org` in the previous section decides that they agree with most of the collaboratively-curated blocklist, but they actually would like to keep federating with ``fashy-arseholes.example.org`` for some godforsaken reason. |
||||
|
||||
This can be done in one of three ways: |
||||
|
||||
1. The admin of `instance-g.example.org` subscribes to the shared blocklist, but they do so with the ["create as drafts"](./settings.md#create-permissions-as-drafts) option set to true. When their instance fetches the blocklist, a draft block is created for `fashy-arseholes.example.org`. The admin of `instance-g` just leaves the permission as a draft, or rejects it, so it never comes into force. |
||||
2. Before the blocklist is re-fetched, the admin of `instance-g.example.org` creates a [domain permission exclude](./settings.md#excludes) entry for ``instance-g.example.org``. The domain ``instance-g.example.org`` then becomes exempt/excluded from automatic permission creation, and so the block for ``instance-g.example.org`` on the shared blocklist does not get created in the database of ``instance-g.example.org`` the next time the list is fetched. |
||||
3. The admin of `instance-g.example.org` creates an explicit domain allow entry for `fashy-arseholes.example.org` on their own instance. Because their instance is running in `blocklist` federation mode, [the explicit allow overrides the domain block entry](./federation_modes.md#in-blocklist-mode), and so the domain remains unblocked. |
||||
|
||||
### 4. Subscribe directly to another instance's blocklist. |
||||
|
||||
Because GoToSocial is able to fetch and parse JSON-formatted lists of domain permissions, it is possible to subscribe directly to another instance's list of blocked domains via their `/api/v1/instance/domain_blocks` (Mastodon) or `/api/v1/instance/peers?filter=suspended` (GoToSocial) endpoint (if exposed). |
||||
|
||||
For example, the Mastodon instance `peepee.poopoo.example.org` exposes their block list publicly, and the owner of the GoToSocial instance `instance-h.example.org` decides they quite like the cut of the Mastodon moderator's jib. They create a domain permission subscription of type JSON, and set the URI to `https://peepee.poopoo.example.org/api/v1/instance/domain_blocks`. Every 24 hours, their instance will go fetch the blocklist JSON from the Mastodon instance, and create permissions based on entries discovered therein. |
||||
|
||||
## Example lists per content type |
||||
|
||||
Shown below are examples of the different permission list formats that GoToSocial is able to understand and parse. |
||||
|
||||
Each list contains three domains, `bumfaces.net`, `peepee.poopoo`, and `nothanks.com`. |
||||
|
||||
### CSV |
||||
|
||||
CSV lists use content type `text/csv`. |
||||
|
||||
Mastodon domain permission exports generally use this format. |
||||
|
||||
```csv |
||||
#domain,#severity,#reject_media,#reject_reports,#public_comment,#obfuscate |
||||
bumfaces.net,suspend,false,false,big jerks,false |
||||
peepee.poopoo,suspend,false,false,harassment,false |
||||
nothanks.com,suspend,false,false,,false |
||||
``` |
||||
|
||||
### JSON (application/json) |
||||
|
||||
JSON lists use content type `application/json`. |
||||
|
||||
```json |
||||
[ |
||||
{ |
||||
"domain": "bumfaces.net", |
||||
"suspended_at": "2020-05-13T13:29:12.000Z", |
||||
"public_comment": "big jerks" |
||||
}, |
||||
{ |
||||
"domain": "peepee.poopoo", |
||||
"suspended_at": "2020-05-13T13:29:12.000Z", |
||||
"public_comment": "harassment" |
||||
}, |
||||
{ |
||||
"domain": "nothanks.com", |
||||
"suspended_at": "2020-05-13T13:29:12.000Z" |
||||
} |
||||
] |
||||
``` |
||||
|
||||
### Plaintext (text/plain) |
||||
|
||||
Plaintext lists use content type `text/plain`. |
||||
|
||||
Note that it is not possible to include any fields like "obfuscate" or "public comment" in plaintext lists, as they are simply a newline-separated list of domains. |
||||
|
||||
```text |
||||
bumfaces.net |
||||
peepee.poopoo |
||||
nothanks.com |
||||
``` |
||||
|
After Width: | Height: | Size: 155 KiB |
|
Before Width: | Height: | Size: 95 KiB After Width: | Height: | Size: 120 KiB |
@ -0,0 +1,51 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package admin |
||||
|
||||
import ( |
||||
"context" |
||||
"time" |
||||
|
||||
"github.com/superseriousbusiness/gotosocial/internal/gtserror" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" |
||||
) |
||||
|
||||
func (a *Actions) DomainKeysExpireF(domain string) ActionF { |
||||
return func(ctx context.Context) gtserror.MultiError { |
||||
var ( |
||||
expiresAt = time.Now() |
||||
errs gtserror.MultiError |
||||
) |
||||
|
||||
// For each account on this domain, expire
|
||||
// the public key and update the account.
|
||||
if err := a.rangeDomainAccounts(ctx, domain, func(account *gtsmodel.Account) { |
||||
account.PublicKeyExpiresAt = expiresAt |
||||
if err := a.db.UpdateAccount(ctx, |
||||
account, |
||||
"public_key_expires_at", |
||||
); err != nil { |
||||
errs.Appendf("db error updating account: %w", err) |
||||
} |
||||
}); err != nil { |
||||
errs.Appendf("db error ranging through accounts: %w", err) |
||||
} |
||||
|
||||
return errs |
||||
} |
||||
} |
||||
@ -0,0 +1,387 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package admin |
||||
|
||||
import ( |
||||
"context" |
||||
"errors" |
||||
"time" |
||||
|
||||
"codeberg.org/gruf/go-kv" |
||||
|
||||
"github.com/superseriousbusiness/gotosocial/internal/ap" |
||||
"github.com/superseriousbusiness/gotosocial/internal/config" |
||||
"github.com/superseriousbusiness/gotosocial/internal/db" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtserror" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" |
||||
"github.com/superseriousbusiness/gotosocial/internal/log" |
||||
"github.com/superseriousbusiness/gotosocial/internal/messages" |
||||
) |
||||
|
||||
// Returns an AdminActionF for
|
||||
// domain allow side effects.
|
||||
func (a *Actions) DomainAllowF( |
||||
actionID string, |
||||
domainAllow *gtsmodel.DomainAllow, |
||||
) ActionF { |
||||
return func(ctx context.Context) gtserror.MultiError { |
||||
l := log. |
||||
WithContext(ctx). |
||||
WithFields(kv.Fields{ |
||||
{"action", "allow"}, |
||||
{"actionID", actionID}, |
||||
{"domain", domainAllow.Domain}, |
||||
}...) |
||||
|
||||
// Log start + finish.
|
||||
l.Info("processing side effects") |
||||
errs := a.domainAllowSideEffects(ctx, domainAllow) |
||||
l.Info("finished processing side effects") |
||||
|
||||
return errs |
||||
} |
||||
} |
||||
|
||||
func (a *Actions) domainAllowSideEffects( |
||||
ctx context.Context, |
||||
allow *gtsmodel.DomainAllow, |
||||
) gtserror.MultiError { |
||||
if config.GetInstanceFederationMode() == config.InstanceFederationModeAllowlist { |
||||
// We're running in allowlist mode,
|
||||
// so there are no side effects to
|
||||
// process here.
|
||||
return nil |
||||
} |
||||
|
||||
// We're running in blocklist mode or
|
||||
// some similar mode which necessitates
|
||||
// domain allow side effects if a block
|
||||
// was in place when the allow was created.
|
||||
//
|
||||
// So, check if there's a block.
|
||||
block, err := a.db.GetDomainBlock(ctx, allow.Domain) |
||||
if err != nil && !errors.Is(err, db.ErrNoEntries) { |
||||
errs := gtserror.NewMultiError(1) |
||||
errs.Appendf("db error getting domain block %s: %w", allow.Domain, err) |
||||
return errs |
||||
} |
||||
|
||||
if block == nil { |
||||
// No block?
|
||||
// No problem!
|
||||
return nil |
||||
} |
||||
|
||||
// There was a block, over which the new
|
||||
// allow ought to take precedence. To account
|
||||
// for this, just run side effects as though
|
||||
// the domain was being unblocked, while
|
||||
// leaving the existing block in place.
|
||||
//
|
||||
// Any accounts that were suspended by
|
||||
// the block will be unsuspended and be
|
||||
// able to interact with the instance again.
|
||||
return a.domainUnblockSideEffects(ctx, block) |
||||
} |
||||
|
||||
// Returns an AdminActionF for
|
||||
// domain unallow side effects.
|
||||
func (a *Actions) DomainUnallowF( |
||||
actionID string, |
||||
domainAllow *gtsmodel.DomainAllow, |
||||
) ActionF { |
||||
return func(ctx context.Context) gtserror.MultiError { |
||||
l := log. |
||||
WithContext(ctx). |
||||
WithFields(kv.Fields{ |
||||
{"action", "unallow"}, |
||||
{"actionID", actionID}, |
||||
{"domain", domainAllow.Domain}, |
||||
}...) |
||||
|
||||
// Log start + finish.
|
||||
l.Info("processing side effects") |
||||
errs := a.domainUnallowSideEffects(ctx, domainAllow) |
||||
l.Info("finished processing side effects") |
||||
|
||||
return errs |
||||
} |
||||
} |
||||
|
||||
func (a *Actions) domainUnallowSideEffects( |
||||
ctx context.Context, |
||||
allow *gtsmodel.DomainAllow, |
||||
) gtserror.MultiError { |
||||
if config.GetInstanceFederationMode() == config.InstanceFederationModeAllowlist { |
||||
// We're running in allowlist mode,
|
||||
// so there are no side effects to
|
||||
// process here.
|
||||
return nil |
||||
} |
||||
|
||||
// We're running in blocklist mode or
|
||||
// some similar mode which necessitates
|
||||
// domain allow side effects if a block
|
||||
// was in place when the allow was removed.
|
||||
//
|
||||
// So, check if there's a block.
|
||||
block, err := a.db.GetDomainBlock(ctx, allow.Domain) |
||||
if err != nil && !errors.Is(err, db.ErrNoEntries) { |
||||
errs := gtserror.NewMultiError(1) |
||||
errs.Appendf("db error getting domain block %s: %w", allow.Domain, err) |
||||
return errs |
||||
} |
||||
|
||||
if block == nil { |
||||
// No block?
|
||||
// No problem!
|
||||
return nil |
||||
} |
||||
|
||||
// There was a block, over which the previous
|
||||
// allow was taking precedence. Now that the
|
||||
// allow has been removed, we should put the
|
||||
// side effects of the block back in place.
|
||||
//
|
||||
// To do this, process the block side effects
|
||||
// again as though the block were freshly
|
||||
// created. This will mark all accounts from
|
||||
// the blocked domain as suspended, and clean
|
||||
// up their follows/following, media, etc.
|
||||
return a.domainBlockSideEffects(ctx, block) |
||||
} |
||||
|
||||
func (a *Actions) DomainBlockF( |
||||
actionID string, |
||||
domainBlock *gtsmodel.DomainBlock, |
||||
) ActionF { |
||||
return func(ctx context.Context) gtserror.MultiError { |
||||
l := log. |
||||
WithContext(ctx). |
||||
WithFields(kv.Fields{ |
||||
{"action", "block"}, |
||||
{"actionID", actionID}, |
||||
{"domain", domainBlock.Domain}, |
||||
}...) |
||||
|
||||
skip, err := a.skipBlockSideEffects(ctx, domainBlock.Domain) |
||||
if err != nil { |
||||
return err |
||||
} |
||||
|
||||
if skip != "" { |
||||
l.Infof("skipping side effects: %s", skip) |
||||
return nil |
||||
} |
||||
|
||||
l.Info("processing side effects") |
||||
errs := a.domainBlockSideEffects(ctx, domainBlock) |
||||
l.Info("finished processing side effects") |
||||
|
||||
return errs |
||||
} |
||||
} |
||||
|
||||
// domainBlockSideEffects processes the side effects of a domain block:
|
||||
//
|
||||
// 1. Strip most info away from the instance entry for the domain.
|
||||
// 2. Pass each account from the domain to the processor for deletion.
|
||||
//
|
||||
// It should be called asynchronously, since it can take a while when
|
||||
// there are many accounts present on the given domain.
|
||||
func (a *Actions) domainBlockSideEffects( |
||||
ctx context.Context, |
||||
block *gtsmodel.DomainBlock, |
||||
) gtserror.MultiError { |
||||
var errs gtserror.MultiError |
||||
|
||||
// If we have an instance entry for this domain,
|
||||
// update it with the new block ID and clear all fields
|
||||
instance, err := a.db.GetInstance(ctx, block.Domain) |
||||
if err != nil && !errors.Is(err, db.ErrNoEntries) { |
||||
errs.Appendf("db error getting instance %s: %w", block.Domain, err) |
||||
return errs |
||||
} |
||||
|
||||
if instance != nil { |
||||
// We had an entry for this domain.
|
||||
columns := stubbifyInstance(instance, block.ID) |
||||
if err := a.db.UpdateInstance(ctx, instance, columns...); err != nil { |
||||
errs.Appendf("db error updating instance: %w", err) |
||||
return errs |
||||
} |
||||
} |
||||
|
||||
// For each account that belongs to this domain,
|
||||
// process an account delete message to remove
|
||||
// that account's posts, media, etc.
|
||||
if err := a.rangeDomainAccounts(ctx, block.Domain, func(account *gtsmodel.Account) { |
||||
if err := a.workers.Client.Process(ctx, &messages.FromClientAPI{ |
||||
APObjectType: ap.ActorPerson, |
||||
APActivityType: ap.ActivityDelete, |
||||
GTSModel: block, |
||||
Origin: account, |
||||
Target: account, |
||||
}); err != nil { |
||||
errs.Append(err) |
||||
} |
||||
}); err != nil { |
||||
errs.Appendf("db error ranging through accounts: %w", err) |
||||
} |
||||
|
||||
return errs |
||||
} |
||||
|
||||
func (a *Actions) DomainUnblockF( |
||||
actionID string, |
||||
domainBlock *gtsmodel.DomainBlock, |
||||
) ActionF { |
||||
return func(ctx context.Context) gtserror.MultiError { |
||||
l := log. |
||||
WithContext(ctx). |
||||
WithFields(kv.Fields{ |
||||
{"action", "unblock"}, |
||||
{"actionID", actionID}, |
||||
{"domain", domainBlock.Domain}, |
||||
}...) |
||||
|
||||
l.Info("processing side effects") |
||||
errs := a.domainUnblockSideEffects(ctx, domainBlock) |
||||
l.Info("finished processing side effects") |
||||
|
||||
return errs |
||||
} |
||||
} |
||||
|
||||
// domainUnblockSideEffects processes the side effects of undoing a
|
||||
// domain block:
|
||||
//
|
||||
// 1. Mark instance entry as no longer suspended.
|
||||
// 2. Mark each account from the domain as no longer suspended, if the
|
||||
// suspension origin corresponds to the ID of the provided domain block.
|
||||
//
|
||||
// It should be called asynchronously, since it can take a while when
|
||||
// there are many accounts present on the given domain.
|
||||
func (a *Actions) domainUnblockSideEffects( |
||||
ctx context.Context, |
||||
block *gtsmodel.DomainBlock, |
||||
) gtserror.MultiError { |
||||
var errs gtserror.MultiError |
||||
|
||||
// Update instance entry for this domain, if we have it.
|
||||
instance, err := a.db.GetInstance(ctx, block.Domain) |
||||
if err != nil && !errors.Is(err, db.ErrNoEntries) { |
||||
errs.Appendf("db error getting instance %s: %w", block.Domain, err) |
||||
} |
||||
|
||||
if instance != nil { |
||||
// We had an entry, update it to signal
|
||||
// that it's no longer suspended.
|
||||
instance.SuspendedAt = time.Time{} |
||||
instance.DomainBlockID = "" |
||||
if err := a.db.UpdateInstance( |
||||
ctx, |
||||
instance, |
||||
"suspended_at", |
||||
"domain_block_id", |
||||
); err != nil { |
||||
errs.Appendf("db error updating instance: %w", err) |
||||
return errs |
||||
} |
||||
} |
||||
|
||||
// Unsuspend all accounts whose suspension origin was this domain block.
|
||||
if err := a.rangeDomainAccounts(ctx, block.Domain, func(account *gtsmodel.Account) { |
||||
if account.SuspensionOrigin == "" || account.SuspendedAt.IsZero() { |
||||
// Account wasn't suspended, nothing to do.
|
||||
return |
||||
} |
||||
|
||||
if account.SuspensionOrigin != block.ID { |
||||
// Account was suspended, but not by
|
||||
// this domain block, leave it alone.
|
||||
return |
||||
} |
||||
|
||||
// Account was suspended by this domain
|
||||
// block, mark it as unsuspended.
|
||||
account.SuspendedAt = time.Time{} |
||||
account.SuspensionOrigin = "" |
||||
|
||||
if err := a.db.UpdateAccount( |
||||
ctx, |
||||
account, |
||||
"suspended_at", |
||||
"suspension_origin", |
||||
); err != nil { |
||||
errs.Appendf("db error updating account %s: %w", account.Username, err) |
||||
} |
||||
}); err != nil { |
||||
errs.Appendf("db error ranging through accounts: %w", err) |
||||
} |
||||
|
||||
return errs |
||||
} |
||||
|
||||
// skipBlockSideEffects checks if side effects of block creation
|
||||
// should be skipped for the given domain, taking account of
|
||||
// instance federation mode, and existence of any allows
|
||||
// which ought to "shield" this domain from being blocked.
|
||||
//
|
||||
// If the caller should skip, the returned string will be non-zero
|
||||
// and will be set to a reason why side effects should be skipped.
|
||||
//
|
||||
// - blocklist mode + allow exists: "..." (skip)
|
||||
// - blocklist mode + no allow: "" (don't skip)
|
||||
// - allowlist mode + allow exists: "" (don't skip)
|
||||
// - allowlist mode + no allow: "" (don't skip)
|
||||
func (a *Actions) skipBlockSideEffects( |
||||
ctx context.Context, |
||||
domain string, |
||||
) (string, gtserror.MultiError) { |
||||
var ( |
||||
skip string // Assume "" (don't skip).
|
||||
errs gtserror.MultiError |
||||
) |
||||
|
||||
// Never skip block side effects in allowlist mode.
|
||||
fediMode := config.GetInstanceFederationMode() |
||||
if fediMode == config.InstanceFederationModeAllowlist { |
||||
return skip, errs |
||||
} |
||||
|
||||
// We know we're in blocklist mode.
|
||||
//
|
||||
// We want to skip domain block side
|
||||
// effects if an allow is already
|
||||
// in place which overrides the block.
|
||||
|
||||
// Check if an explicit allow exists for this domain.
|
||||
domainAllow, err := a.db.GetDomainAllow(ctx, domain) |
||||
if err != nil && !errors.Is(err, db.ErrNoEntries) { |
||||
errs.Appendf("error getting domain allow: %w", err) |
||||
return skip, errs |
||||
} |
||||
|
||||
if domainAllow != nil { |
||||
skip = "running in blocklist mode, and an explicit allow exists for this domain" |
||||
return skip, errs |
||||
} |
||||
|
||||
return skip, errs |
||||
} |
||||
@ -0,0 +1,99 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package admin |
||||
|
||||
import ( |
||||
"context" |
||||
"errors" |
||||
"time" |
||||
|
||||
"github.com/superseriousbusiness/gotosocial/internal/db" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtserror" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" |
||||
) |
||||
|
||||
// stubbifyInstance renders the given instance as a stub,
|
||||
// removing most information from it and marking it as
|
||||
// suspended.
|
||||
//
|
||||
// For caller's convenience, this function returns the db
|
||||
// names of all columns that are updated by it.
|
||||
func stubbifyInstance(instance *gtsmodel.Instance, domainBlockID string) []string { |
||||
instance.Title = "" |
||||
instance.SuspendedAt = time.Now() |
||||
instance.DomainBlockID = domainBlockID |
||||
instance.ShortDescription = "" |
||||
instance.Description = "" |
||||
instance.Terms = "" |
||||
instance.ContactEmail = "" |
||||
instance.ContactAccountUsername = "" |
||||
instance.ContactAccountID = "" |
||||
instance.Version = "" |
||||
|
||||
return []string{ |
||||
"title", |
||||
"suspended_at", |
||||
"domain_block_id", |
||||
"short_description", |
||||
"description", |
||||
"terms", |
||||
"contact_email", |
||||
"contact_account_username", |
||||
"contact_account_id", |
||||
"version", |
||||
} |
||||
} |
||||
|
||||
// rangeDomainAccounts iterates through all accounts
|
||||
// originating from the given domain, and calls the
|
||||
// provided range function on each account.
|
||||
//
|
||||
// If an error is returned while selecting accounts,
|
||||
// the loop will stop and return the error.
|
||||
func (a *Actions) rangeDomainAccounts( |
||||
ctx context.Context, |
||||
domain string, |
||||
rangeF func(*gtsmodel.Account), |
||||
) error { |
||||
var ( |
||||
limit = 50 // Limit selection to avoid spiking mem/cpu.
|
||||
maxID string // Start with empty string to select from top.
|
||||
) |
||||
|
||||
for { |
||||
// Get (next) page of accounts.
|
||||
accounts, err := a.db.GetInstanceAccounts(ctx, domain, maxID, limit) |
||||
if err != nil && !errors.Is(err, db.ErrNoEntries) { |
||||
// Real db error.
|
||||
return gtserror.Newf("db error getting instance accounts: %w", err) |
||||
} |
||||
|
||||
if len(accounts) == 0 { |
||||
// No accounts left, we're done.
|
||||
return nil |
||||
} |
||||
|
||||
// Set next max ID for paging down.
|
||||
maxID = accounts[len(accounts)-1].ID |
||||
|
||||
// Call provided range function.
|
||||
for _, account := range accounts { |
||||
rangeF(account) |
||||
} |
||||
} |
||||
} |
||||
@ -0,0 +1,118 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package admin |
||||
|
||||
import ( |
||||
"fmt" |
||||
"net/http" |
||||
|
||||
"github.com/gin-gonic/gin" |
||||
apiutil "github.com/superseriousbusiness/gotosocial/internal/api/util" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtserror" |
||||
"github.com/superseriousbusiness/gotosocial/internal/oauth" |
||||
) |
||||
|
||||
// DomainPermissionSubscriptionTestPOSTHandler swagger:operation POST /api/v1/admin/domain_permission_subscriptions/{id}/test domainPermissionSubscriptionTest
|
||||
//
|
||||
// Test one domain permission subscription by making your instance fetch and parse it *without creating permissions*.
|
||||
//
|
||||
// The response body will be a list of domain permissions that *would* be created by this subscription, OR an error message.
|
||||
//
|
||||
// This is useful in cases where you want to check that your instance can actually fetch + parse a list.
|
||||
//
|
||||
// ---
|
||||
// tags:
|
||||
// - admin
|
||||
//
|
||||
// produces:
|
||||
// - application/json
|
||||
//
|
||||
// parameters:
|
||||
// -
|
||||
// name: id
|
||||
// required: true
|
||||
// in: path
|
||||
// description: ID of the domain permission draft.
|
||||
// type: string
|
||||
//
|
||||
// security:
|
||||
// - OAuth2 Bearer:
|
||||
// - admin
|
||||
//
|
||||
// responses:
|
||||
// '200':
|
||||
// description: >-
|
||||
// Either an array of domain permissions, OR an error message of the form
|
||||
// `{"error":"[ERROR MESSAGE HERE]"}` indicating why the list could not be fetched.
|
||||
// schema:
|
||||
// type: array
|
||||
// items:
|
||||
// "$ref": "#/definitions/domain"
|
||||
// '400':
|
||||
// description: bad request
|
||||
// '401':
|
||||
// description: unauthorized
|
||||
// '403':
|
||||
// description: forbidden
|
||||
// '406':
|
||||
// description: not acceptable
|
||||
// '409':
|
||||
// description: conflict
|
||||
// '500':
|
||||
// description: internal server error
|
||||
func (m *Module) DomainPermissionSubscriptionTestPOSTHandler(c *gin.Context) { |
||||
authed, err := oauth.Authed(c, true, true, true, true) |
||||
if err != nil { |
||||
apiutil.ErrorHandler(c, gtserror.NewErrorUnauthorized(err, err.Error()), m.processor.InstanceGetV1) |
||||
return |
||||
} |
||||
|
||||
if !*authed.User.Admin { |
||||
err := fmt.Errorf("user %s not an admin", authed.User.ID) |
||||
apiutil.ErrorHandler(c, gtserror.NewErrorForbidden(err, err.Error()), m.processor.InstanceGetV1) |
||||
return |
||||
} |
||||
|
||||
if authed.Account.IsMoving() { |
||||
apiutil.ForbiddenAfterMove(c) |
||||
return |
||||
} |
||||
|
||||
if _, err := apiutil.NegotiateAccept(c, apiutil.JSONAcceptHeaders...); err != nil { |
||||
apiutil.ErrorHandler(c, gtserror.NewErrorNotAcceptable(err, err.Error()), m.processor.InstanceGetV1) |
||||
return |
||||
} |
||||
|
||||
id, errWithCode := apiutil.ParseID(c.Param(apiutil.IDKey)) |
||||
if errWithCode != nil { |
||||
apiutil.ErrorHandler(c, errWithCode, m.processor.InstanceGetV1) |
||||
return |
||||
} |
||||
|
||||
resp, errWithCode := m.processor.Admin().DomainPermissionSubscriptionTest( |
||||
c.Request.Context(), |
||||
authed.Account, |
||||
id, |
||||
) |
||||
if errWithCode != nil { |
||||
apiutil.ErrorHandler(c, errWithCode, m.processor.InstanceGetV1) |
||||
return |
||||
} |
||||
|
||||
apiutil.JSON(c, http.StatusOK, resp) |
||||
} |
||||
@ -0,0 +1,125 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package admin_test |
||||
|
||||
import ( |
||||
"bytes" |
||||
"context" |
||||
"encoding/json" |
||||
"io" |
||||
"net/http" |
||||
"net/http/httptest" |
||||
"strings" |
||||
"testing" |
||||
|
||||
"github.com/gin-gonic/gin" |
||||
"github.com/stretchr/testify/suite" |
||||
"github.com/superseriousbusiness/gotosocial/internal/api/client/admin" |
||||
apiutil "github.com/superseriousbusiness/gotosocial/internal/api/util" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" |
||||
"github.com/superseriousbusiness/gotosocial/internal/util" |
||||
) |
||||
|
||||
type DomainPermissionSubscriptionTestTestSuite struct { |
||||
AdminStandardTestSuite |
||||
} |
||||
|
||||
func (suite *DomainPermissionSubscriptionTestTestSuite) TestDomainPermissionSubscriptionTest() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
permSub = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/baddies.csv", |
||||
ContentType: gtsmodel.DomainPermSubContentTypeCSV, |
||||
} |
||||
) |
||||
|
||||
// Create a subscription for a CSV list of baddies.
|
||||
err := suite.state.DB.PutDomainPermissionSubscription(ctx, permSub) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Prepare the request to the /test endpoint.
|
||||
subPath := strings.ReplaceAll( |
||||
admin.DomainPermissionSubscriptionTestPath, |
||||
":id", permSub.ID, |
||||
) |
||||
path := "/api" + subPath |
||||
recorder := httptest.NewRecorder() |
||||
ginCtx := suite.newContext(recorder, http.MethodPost, nil, path, "application/json") |
||||
ginCtx.Params = gin.Params{ |
||||
gin.Param{ |
||||
Key: apiutil.IDKey, |
||||
Value: permSub.ID, |
||||
}, |
||||
} |
||||
|
||||
// Trigger the handler.
|
||||
suite.adminModule.DomainPermissionSubscriptionTestPOSTHandler(ginCtx) |
||||
suite.Equal(http.StatusOK, recorder.Code) |
||||
|
||||
// Read the body back.
|
||||
b, err := io.ReadAll(recorder.Body) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
dst := new(bytes.Buffer) |
||||
if err := json.Indent(dst, b, "", " "); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Ensure expected.
|
||||
suite.Equal(`[ |
||||
{ |
||||
"domain": "bumfaces.net", |
||||
"public_comment": "big jerks" |
||||
}, |
||||
{ |
||||
"domain": "peepee.poopoo", |
||||
"public_comment": "harassment" |
||||
}, |
||||
{ |
||||
"domain": "nothanks.com" |
||||
} |
||||
]`, dst.String()) |
||||
|
||||
// No permissions should be created
|
||||
// since this is a dry run / test.
|
||||
blocked, err := suite.state.DB.AreDomainsBlocked( |
||||
ctx, |
||||
[]string{"bumfaces.net", "peepee.poopoo", "nothanks.com"}, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
suite.False(blocked) |
||||
} |
||||
|
||||
func TestDomainPermissionSubscriptionTestTestSuite(t *testing.T) { |
||||
suite.Run(t, &DomainPermissionSubscriptionTestTestSuite{}) |
||||
} |
||||
@ -0,0 +1,811 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package subscriptions |
||||
|
||||
import ( |
||||
"bufio" |
||||
"context" |
||||
"encoding/csv" |
||||
"encoding/json" |
||||
"errors" |
||||
"fmt" |
||||
"io" |
||||
"slices" |
||||
"strconv" |
||||
"strings" |
||||
"time" |
||||
|
||||
"codeberg.org/gruf/go-kv" |
||||
|
||||
"github.com/miekg/dns" |
||||
"github.com/superseriousbusiness/gotosocial/internal/admin" |
||||
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" |
||||
"github.com/superseriousbusiness/gotosocial/internal/config" |
||||
"github.com/superseriousbusiness/gotosocial/internal/db" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtserror" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" |
||||
"github.com/superseriousbusiness/gotosocial/internal/id" |
||||
"github.com/superseriousbusiness/gotosocial/internal/log" |
||||
"github.com/superseriousbusiness/gotosocial/internal/transport" |
||||
"github.com/superseriousbusiness/gotosocial/internal/util" |
||||
) |
||||
|
||||
// ScheduleJobs schedules domain permission subscription
|
||||
// fetching + updating using configured parameters.
|
||||
//
|
||||
// Returns an error if `MediaCleanupFrom`
|
||||
// is not a valid format (hh:mm:ss).
|
||||
func (s *Subscriptions) ScheduleJobs() error { |
||||
const hourMinute = "15:04" |
||||
|
||||
var ( |
||||
now = time.Now() |
||||
processEvery = config.GetInstanceSubscriptionsProcessEvery() |
||||
processFromStr = config.GetInstanceSubscriptionsProcessFrom() |
||||
) |
||||
|
||||
// Parse processFromStr as hh:mm.
|
||||
// Resulting time will be on 1 Jan year zero.
|
||||
processFrom, err := time.Parse(hourMinute, processFromStr) |
||||
if err != nil { |
||||
return gtserror.Newf( |
||||
"error parsing '%s' in time format 'hh:mm': %w", |
||||
processFromStr, err, |
||||
) |
||||
} |
||||
|
||||
// Time travel from
|
||||
// year zero, groovy.
|
||||
firstProcessAt := time.Date( |
||||
now.Year(), |
||||
now.Month(), |
||||
now.Day(), |
||||
processFrom.Hour(), |
||||
processFrom.Minute(), |
||||
0, |
||||
0, |
||||
now.Location(), |
||||
) |
||||
|
||||
// Ensure first processing is in the future.
|
||||
for firstProcessAt.Before(now) { |
||||
firstProcessAt = firstProcessAt.Add(processEvery) |
||||
} |
||||
|
||||
fn := func(ctx context.Context, start time.Time) { |
||||
log.Info(ctx, "starting instance subscriptions processing") |
||||
|
||||
// In blocklist (default) mode, process allows
|
||||
// first to provide immunity to block side effects.
|
||||
//
|
||||
// In allowlist mode, process blocks first to
|
||||
// ensure allowlist doesn't override blocks.
|
||||
var order [2]gtsmodel.DomainPermissionType |
||||
if config.GetInstanceFederationMode() == config.InstanceFederationModeBlocklist { |
||||
order = [2]gtsmodel.DomainPermissionType{ |
||||
gtsmodel.DomainPermissionAllow, |
||||
gtsmodel.DomainPermissionBlock, |
||||
} |
||||
} else { |
||||
order = [2]gtsmodel.DomainPermissionType{ |
||||
gtsmodel.DomainPermissionBlock, |
||||
gtsmodel.DomainPermissionAllow, |
||||
} |
||||
} |
||||
|
||||
// Fetch + process subscribed perms in order.
|
||||
for _, permType := range order { |
||||
s.ProcessDomainPermissionSubscriptions(ctx, permType) |
||||
} |
||||
|
||||
log.Infof(ctx, "finished instance subscriptions processing after %s", time.Since(start)) |
||||
} |
||||
|
||||
log.Infof(nil, |
||||
"scheduling instance subscriptions processing to run every %s, starting from %s; next processing will run at %s", |
||||
processEvery, processFromStr, firstProcessAt, |
||||
) |
||||
|
||||
// Schedule processing to execute according to schedule.
|
||||
if !s.state.Workers.Scheduler.AddRecurring( |
||||
"@subsprocessing", |
||||
firstProcessAt, |
||||
processEvery, |
||||
fn, |
||||
) { |
||||
panic("failed to schedule @subsprocessing") |
||||
} |
||||
|
||||
return nil |
||||
} |
||||
|
||||
// ProcessDomainPermissionSubscriptions processes all domain permission
|
||||
// subscriptions of the given permission type by, in turn, calling the
|
||||
// URI of each subscription, parsing the result into a list of domain
|
||||
// permissions, and creating (or skipping) each permission as appropriate.
|
||||
func (s *Subscriptions) ProcessDomainPermissionSubscriptions( |
||||
ctx context.Context, |
||||
permType gtsmodel.DomainPermissionType, |
||||
) { |
||||
log.Info(ctx, "start") |
||||
defer log.Info(ctx, "finished") |
||||
|
||||
// Get permission subscriptions in priority order (highest -> lowest).
|
||||
permSubs, err := s.state.DB.GetDomainPermissionSubscriptionsByPriority(ctx, permType) |
||||
if err != nil && !errors.Is(err, db.ErrNoEntries) { |
||||
// Real db error.
|
||||
log.Errorf(ctx, "db error getting domain perm subs by priority: %v", err) |
||||
return |
||||
} |
||||
|
||||
if len(permSubs) == 0 { |
||||
// No subscriptions of this
|
||||
// type, so nothing to do.
|
||||
return |
||||
} |
||||
|
||||
// Get a transport using the instance account,
|
||||
// we can reuse this for each HTTP call.
|
||||
tsport, err := s.transportController.NewTransportForUsername(ctx, "") |
||||
if err != nil { |
||||
log.Errorf(ctx, "error getting transport for instance account: %v", err) |
||||
return |
||||
} |
||||
|
||||
for i, permSub := range permSubs { |
||||
// Higher priority permission subs = everything
|
||||
// above this permission sub in the slice.
|
||||
higherPrios := permSubs[:i] |
||||
|
||||
_, err := s.ProcessDomainPermissionSubscription( |
||||
ctx, |
||||
permSub, |
||||
tsport, |
||||
higherPrios, |
||||
false, // Not dry. Wet, if you will.
|
||||
) |
||||
if err != nil { |
||||
// Real db error.
|
||||
log.Errorf(ctx, |
||||
"error processing domain permission subscription %s: %v", |
||||
permSub.URI, err, |
||||
) |
||||
return |
||||
} |
||||
|
||||
// Update this perm sub.
|
||||
err = s.state.DB.UpdateDomainPermissionSubscription(ctx, permSub) |
||||
if err != nil { |
||||
// Real db error.
|
||||
log.Errorf(ctx, "db error updating domain perm sub: %v", err) |
||||
return |
||||
} |
||||
} |
||||
} |
||||
|
||||
// ProcessDomainPermissionSubscription processes one domain permission
|
||||
// subscription by dereferencing the URI, parsing the response into a list
|
||||
// of permissions, and for each discovered permission either creating an
|
||||
// entry in the database, or ignoring it if it's excluded or already
|
||||
// covered by a higher-priority subscription.
|
||||
//
|
||||
// On success, the slice of discovered DomainPermissions will be returned.
|
||||
// In case of parsing error, or error on the remote side, permSub.Error
|
||||
// will be updated with the calling/parsing error, and `nil, nil` will be
|
||||
// returned. In case of an actual db error, `nil, err` will be returned and
|
||||
// the caller should handle it.
|
||||
//
|
||||
// getHigherPrios should be a function for returning a slice of domain
|
||||
// permission subscriptions with a higher priority than the given permSub.
|
||||
//
|
||||
// If dry == true, then the URI will still be called, and permissions
|
||||
// will be parsed, but they will not actually be created.
|
||||
//
|
||||
// Note that while this function modifies fields on the given permSub,
|
||||
// it's up to the caller to update it in the database (if desired).
|
||||
func (s *Subscriptions) ProcessDomainPermissionSubscription( |
||||
ctx context.Context, |
||||
permSub *gtsmodel.DomainPermissionSubscription, |
||||
tsport transport.Transport, |
||||
higherPrios []*gtsmodel.DomainPermissionSubscription, |
||||
dry bool, |
||||
) ([]gtsmodel.DomainPermission, error) { |
||||
l := log. |
||||
WithContext(ctx). |
||||
WithFields(kv.Fields{ |
||||
{"permType", permSub.PermissionType.String()}, |
||||
{"permSubURI", permSub.URI}, |
||||
}...) |
||||
|
||||
// Set FetchedAt as we're
|
||||
// going to attempt this now.
|
||||
permSub.FetchedAt = time.Now() |
||||
|
||||
// Call the URI, and only skip
|
||||
// cache if we're doing a dry run.
|
||||
resp, err := tsport.DereferenceDomainPermissions( |
||||
ctx, permSub, dry, |
||||
) |
||||
if err != nil { |
||||
// Couldn't get this one,
|
||||
// set error + return.
|
||||
errStr := err.Error() |
||||
l.Warnf("couldn't dereference permSubURI: %+v", err) |
||||
permSub.Error = errStr |
||||
return nil, nil |
||||
} |
||||
|
||||
// If the permissions at URI weren't modified
|
||||
// since last time, just update some metadata
|
||||
// to indicate a successful fetch, and return.
|
||||
if resp.Unmodified { |
||||
l.Debug("received 304 Not Modified from remote") |
||||
permSub.SuccessfullyFetchedAt = permSub.FetchedAt |
||||
if permSub.ETag == "" && resp.ETag != "" { |
||||
// We didn't have an ETag before but
|
||||
// we have one now: probably the remote
|
||||
// added ETag support in the meantime.
|
||||
permSub.ETag = resp.ETag |
||||
} |
||||
return nil, nil |
||||
} |
||||
|
||||
// At this point we know we got a 200 OK
|
||||
// from the URI, so we've got a live body!
|
||||
// Try to parse the body as a list of wantedPerms
|
||||
// that the subscription wants to create.
|
||||
var wantedPerms []gtsmodel.DomainPermission |
||||
|
||||
switch permSub.ContentType { |
||||
|
||||
// text/csv
|
||||
case gtsmodel.DomainPermSubContentTypeCSV: |
||||
wantedPerms, err = permsFromCSV(l, permSub.PermissionType, resp.Body) |
||||
|
||||
// application/json
|
||||
case gtsmodel.DomainPermSubContentTypeJSON: |
||||
wantedPerms, err = permsFromJSON(l, permSub.PermissionType, resp.Body) |
||||
|
||||
// text/plain
|
||||
case gtsmodel.DomainPermSubContentTypePlain: |
||||
wantedPerms, err = permsFromPlain(l, permSub.PermissionType, resp.Body) |
||||
} |
||||
|
||||
if err != nil { |
||||
// We retrieved the permissions from remote but
|
||||
// the connection died halfway through transfer,
|
||||
// or we couldn't parse the results, or something.
|
||||
// Just set error and return.
|
||||
errStr := err.Error() |
||||
l.Warnf("couldn't parse results: %+v", err) |
||||
permSub.Error = errStr |
||||
return nil, nil |
||||
} |
||||
|
||||
if len(wantedPerms) == 0 { |
||||
// Fetch was OK, and parsing was, on the surface at
|
||||
// least, OK, but we didn't get any perms. Consider
|
||||
// this an error as users will probably want to know.
|
||||
const errStr = "fetch successful but parsed zero usable results" |
||||
l.Warn(errStr) |
||||
permSub.Error = errStr |
||||
return nil, nil |
||||
} |
||||
|
||||
// This can now be considered a successful fetch.
|
||||
permSub.SuccessfullyFetchedAt = permSub.FetchedAt |
||||
permSub.ETag = resp.ETag |
||||
permSub.Error = "" |
||||
|
||||
// Keep track of which domain perms are
|
||||
// created (or would be, if dry == true).
|
||||
createdPerms := make([]gtsmodel.DomainPermission, 0, len(wantedPerms)) |
||||
|
||||
// Iterate through wantedPerms and
|
||||
// create (or dry create) each one.
|
||||
for _, wantedPerm := range wantedPerms { |
||||
l = l.WithField("domain", wantedPerm.GetDomain()) |
||||
created, err := s.processDomainPermission( |
||||
ctx, l, |
||||
wantedPerm, |
||||
permSub, |
||||
higherPrios, |
||||
dry, |
||||
) |
||||
if err != nil { |
||||
// Proper db error.
|
||||
return nil, err |
||||
} |
||||
|
||||
if !created { |
||||
continue |
||||
} |
||||
|
||||
createdPerms = append(createdPerms, wantedPerm) |
||||
} |
||||
|
||||
return createdPerms, nil |
||||
} |
||||
|
||||
// processDomainPermission processes one wanted domain
|
||||
// permission discovered via a domain permission sub's URI.
|
||||
//
|
||||
// Error will only be returned in case of an actual database
|
||||
// error, else the error will be logged and nil returned.
|
||||
func (s *Subscriptions) processDomainPermission( |
||||
ctx context.Context, |
||||
l log.Entry, |
||||
wantedPerm gtsmodel.DomainPermission, |
||||
permSub *gtsmodel.DomainPermissionSubscription, |
||||
higherPrios []*gtsmodel.DomainPermissionSubscription, |
||||
dry bool, |
||||
) (bool, error) { |
||||
// Set to true if domain permission
|
||||
// actually (would be) created.
|
||||
var created bool |
||||
|
||||
// If domain is excluded from automatic
|
||||
// permission creation, don't process it.
|
||||
domain := wantedPerm.GetDomain() |
||||
excluded, err := s.state.DB.IsDomainPermissionExcluded(ctx, domain) |
||||
if err != nil { |
||||
// Proper db error.
|
||||
return created, err |
||||
} |
||||
|
||||
if excluded { |
||||
l.Debug("domain is excluded, skipping") |
||||
return created, nil |
||||
} |
||||
|
||||
// Check if a permission already exists for
|
||||
// this domain, and if it's covered already
|
||||
// by a higher-priority subscription.
|
||||
existingPerm, covered, err := s.existingCovered( |
||||
ctx, permSub.PermissionType, domain, higherPrios, |
||||
) |
||||
if err != nil { |
||||
// Proper db error.
|
||||
return created, err |
||||
} |
||||
|
||||
if covered { |
||||
l.Debug("domain is covered by a higher-priority subscription, skipping") |
||||
return created, nil |
||||
} |
||||
|
||||
// At this point we know we
|
||||
// should create the perm.
|
||||
created = true |
||||
|
||||
if dry { |
||||
// Don't do creation or side
|
||||
// effects if we're dry running.
|
||||
return created, nil |
||||
} |
||||
|
||||
// Handle perm creation differently depending
|
||||
// on whether or not a perm already existed.
|
||||
existing := !util.IsNil(existingPerm) |
||||
switch { |
||||
|
||||
case !existing && *permSub.AsDraft: |
||||
// No existing perm, create as draft.
|
||||
err = s.state.DB.PutDomainPermissionDraft( |
||||
ctx, |
||||
>smodel.DomainPermissionDraft{ |
||||
ID: id.NewULID(), |
||||
PermissionType: permSub.PermissionType, |
||||
Domain: domain, |
||||
CreatedByAccountID: permSub.CreatedByAccount.ID, |
||||
CreatedByAccount: permSub.CreatedByAccount, |
||||
PrivateComment: permSub.URI, |
||||
PublicComment: wantedPerm.GetPublicComment(), |
||||
Obfuscate: wantedPerm.GetObfuscate(), |
||||
SubscriptionID: permSub.ID, |
||||
}, |
||||
) |
||||
|
||||
case !existing && !*permSub.AsDraft: |
||||
// No existing perm, create a new one of the
|
||||
// appropriate type, and process side effects.
|
||||
var ( |
||||
insertF func() error |
||||
action *gtsmodel.AdminAction |
||||
actionF admin.ActionF |
||||
) |
||||
|
||||
if permSub.PermissionType == gtsmodel.DomainPermissionBlock { |
||||
// Prepare to insert + process a block.
|
||||
domainBlock := >smodel.DomainBlock{ |
||||
ID: id.NewULID(), |
||||
Domain: domain, |
||||
CreatedByAccountID: permSub.CreatedByAccount.ID, |
||||
CreatedByAccount: permSub.CreatedByAccount, |
||||
PrivateComment: permSub.URI, |
||||
PublicComment: wantedPerm.GetPublicComment(), |
||||
Obfuscate: wantedPerm.GetObfuscate(), |
||||
SubscriptionID: permSub.ID, |
||||
} |
||||
insertF = func() error { return s.state.DB.CreateDomainBlock(ctx, domainBlock) } |
||||
|
||||
action = >smodel.AdminAction{ |
||||
ID: id.NewULID(), |
||||
TargetCategory: gtsmodel.AdminActionCategoryDomain, |
||||
TargetID: domain, |
||||
Type: gtsmodel.AdminActionSuspend, |
||||
AccountID: permSub.CreatedByAccountID, |
||||
} |
||||
actionF = s.state.AdminActions.DomainBlockF(action.ID, domainBlock) |
||||
|
||||
} else { |
||||
// Prepare to insert + process an allow.
|
||||
domainAllow := >smodel.DomainAllow{ |
||||
ID: id.NewULID(), |
||||
Domain: domain, |
||||
CreatedByAccountID: permSub.CreatedByAccount.ID, |
||||
CreatedByAccount: permSub.CreatedByAccount, |
||||
PrivateComment: permSub.URI, |
||||
PublicComment: wantedPerm.GetPublicComment(), |
||||
Obfuscate: wantedPerm.GetObfuscate(), |
||||
SubscriptionID: permSub.ID, |
||||
} |
||||
insertF = func() error { return s.state.DB.CreateDomainAllow(ctx, domainAllow) } |
||||
|
||||
action = >smodel.AdminAction{ |
||||
ID: id.NewULID(), |
||||
TargetCategory: gtsmodel.AdminActionCategoryDomain, |
||||
TargetID: domain, |
||||
Type: gtsmodel.AdminActionUnsuspend, |
||||
AccountID: permSub.CreatedByAccountID, |
||||
} |
||||
actionF = s.state.AdminActions.DomainAllowF(action.ID, domainAllow) |
||||
} |
||||
|
||||
// Insert the new perm in the db.
|
||||
if err = insertF(); err != nil { |
||||
// Couldn't insert wanted perm,
|
||||
// don't process side effects.
|
||||
break |
||||
} |
||||
|
||||
// Run admin action to process
|
||||
// side effects of permission.
|
||||
err = s.state.AdminActions.Run(ctx, action, actionF) |
||||
|
||||
case existingPerm.GetSubscriptionID() != "" || *permSub.AdoptOrphans: |
||||
// Perm exists but we should adopt/take
|
||||
// it by copying over desired fields.
|
||||
existingPerm.SetCreatedByAccountID(wantedPerm.GetCreatedByAccountID()) |
||||
existingPerm.SetCreatedByAccount(wantedPerm.GetCreatedByAccount()) |
||||
existingPerm.SetSubscriptionID(permSub.ID) |
||||
existingPerm.SetObfuscate(wantedPerm.GetObfuscate()) |
||||
existingPerm.SetPrivateComment(wantedPerm.GetPrivateComment()) |
||||
existingPerm.SetPublicComment(wantedPerm.GetPublicComment()) |
||||
|
||||
switch p := existingPerm.(type) { |
||||
case *gtsmodel.DomainBlock: |
||||
err = s.state.DB.UpdateDomainBlock(ctx, p) |
||||
case *gtsmodel.DomainAllow: |
||||
err = s.state.DB.UpdateDomainAllow(ctx, p) |
||||
} |
||||
|
||||
default: |
||||
// Perm exists but we should leave it alone.
|
||||
l.Debug("domain is covered by a higher-priority subscription, skipping") |
||||
} |
||||
|
||||
if err != nil && !errors.Is(err, db.ErrAlreadyExists) { |
||||
// Proper db error.
|
||||
return created, err |
||||
} |
||||
|
||||
created = true |
||||
return created, nil |
||||
} |
||||
|
||||
func permsFromCSV( |
||||
l log.Entry, |
||||
permType gtsmodel.DomainPermissionType, |
||||
body io.ReadCloser, |
||||
) ([]gtsmodel.DomainPermission, error) { |
||||
csvReader := csv.NewReader(body) |
||||
|
||||
// Read and validate column headers.
|
||||
columnHeaders, err := csvReader.Read() |
||||
if err != nil { |
||||
body.Close() |
||||
return nil, gtserror.NewfAt(3, "error decoding csv column headers: %w", err) |
||||
} |
||||
|
||||
if !slices.Equal( |
||||
columnHeaders, |
||||
[]string{ |
||||
"#domain", |
||||
"#severity", |
||||
"#reject_media", |
||||
"#reject_reports", |
||||
"#public_comment", |
||||
"#obfuscate", |
||||
}, |
||||
) { |
||||
body.Close() |
||||
err := gtserror.NewfAt(3, "unexpected column headers in csv: %+v", columnHeaders) |
||||
return nil, err |
||||
} |
||||
|
||||
// Read remaining CSV records.
|
||||
records, err := csvReader.ReadAll() |
||||
|
||||
// Totally done
|
||||
// with body now.
|
||||
body.Close() |
||||
|
||||
// Check for decode error.
|
||||
if err != nil { |
||||
err := gtserror.NewfAt(3, "error decoding body into csv: %w", err) |
||||
return nil, err |
||||
} |
||||
|
||||
// Make sure we actually
|
||||
// have some records.
|
||||
if len(records) == 0 { |
||||
return nil, nil |
||||
} |
||||
|
||||
// Convert records to permissions slice.
|
||||
perms := make([]gtsmodel.DomainPermission, 0, len(records)) |
||||
for _, record := range records { |
||||
if len(record) != 6 { |
||||
l.Warnf("skipping invalid-length record: %+v", record) |
||||
continue |
||||
} |
||||
|
||||
var ( |
||||
domainRaw = record[0] |
||||
severity = record[1] |
||||
publicComment = record[4] |
||||
obfuscateStr = record[5] |
||||
) |
||||
|
||||
if severity != "suspend" { |
||||
l.Warnf("skipping non-suspend record: %+v", record) |
||||
continue |
||||
} |
||||
|
||||
obfuscate, err := strconv.ParseBool(obfuscateStr) |
||||
if err != nil { |
||||
l.Warnf("couldn't parse obfuscate field of record: %+v", record) |
||||
continue |
||||
} |
||||
|
||||
// Normalize + validate domain.
|
||||
domain, err := validateDomain(domainRaw) |
||||
if err != nil { |
||||
l.Warnf("skipping invalid domain %s: %+v", domainRaw, err) |
||||
continue |
||||
} |
||||
|
||||
// Instantiate the permission
|
||||
// as either block or allow.
|
||||
var perm gtsmodel.DomainPermission |
||||
switch permType { |
||||
case gtsmodel.DomainPermissionBlock: |
||||
perm = >smodel.DomainBlock{Domain: domain} |
||||
case gtsmodel.DomainPermissionAllow: |
||||
perm = >smodel.DomainAllow{Domain: domain} |
||||
} |
||||
|
||||
// Set remaining fields.
|
||||
perm.SetPublicComment(publicComment) |
||||
perm.SetObfuscate(&obfuscate) |
||||
|
||||
// We're done.
|
||||
perms = append(perms, perm) |
||||
} |
||||
|
||||
return perms, nil |
||||
} |
||||
|
||||
func permsFromJSON( |
||||
l log.Entry, |
||||
permType gtsmodel.DomainPermissionType, |
||||
body io.ReadCloser, |
||||
) ([]gtsmodel.DomainPermission, error) { |
||||
var ( |
||||
dec = json.NewDecoder(body) |
||||
apiPerms = make([]*apimodel.DomainPermission, 0) |
||||
) |
||||
|
||||
// Read body into memory as
|
||||
// slice of domain permissions.
|
||||
if err := dec.Decode(&apiPerms); err != nil { |
||||
_ = body.Close() // ensure closed.
|
||||
return nil, gtserror.NewfAt(3, "error decoding into json: %w", err) |
||||
} |
||||
|
||||
// Perform a secondary decode just to ensure we drained the
|
||||
// entirety of the data source. Error indicates either extra
|
||||
// trailing garbage, or multiple JSON values (invalid data).
|
||||
if err := dec.Decode(&struct{}{}); err != io.EOF { |
||||
_ = body.Close() // ensure closed.
|
||||
return nil, gtserror.NewfAt(3, "data remaining after json") |
||||
} |
||||
|
||||
// Done with body.
|
||||
_ = body.Close() |
||||
|
||||
// Convert apimodel perms to barebones internal perms.
|
||||
perms := make([]gtsmodel.DomainPermission, 0, len(apiPerms)) |
||||
for _, apiPerm := range apiPerms { |
||||
|
||||
// Normalize + validate domain.
|
||||
domainRaw := apiPerm.Domain.Domain |
||||
domain, err := validateDomain(domainRaw) |
||||
if err != nil { |
||||
l.Warnf("skipping invalid domain %s: %+v", domainRaw, err) |
||||
continue |
||||
} |
||||
|
||||
// Instantiate the permission
|
||||
// as either block or allow.
|
||||
var perm gtsmodel.DomainPermission |
||||
switch permType { |
||||
case gtsmodel.DomainPermissionBlock: |
||||
perm = >smodel.DomainBlock{Domain: domain} |
||||
case gtsmodel.DomainPermissionAllow: |
||||
perm = >smodel.DomainAllow{Domain: domain} |
||||
} |
||||
|
||||
// Set remaining fields.
|
||||
perm.SetPublicComment(apiPerm.PublicComment) |
||||
perm.SetObfuscate(&apiPerm.Obfuscate) |
||||
|
||||
// We're done.
|
||||
perms = append(perms, perm) |
||||
} |
||||
|
||||
return perms, nil |
||||
} |
||||
|
||||
func permsFromPlain( |
||||
l log.Entry, |
||||
permType gtsmodel.DomainPermissionType, |
||||
body io.ReadCloser, |
||||
) ([]gtsmodel.DomainPermission, error) { |
||||
// Scan + split by line.
|
||||
sc := bufio.NewScanner(body) |
||||
|
||||
// Read into domains
|
||||
// line by line.
|
||||
var domains []string |
||||
for sc.Scan() { |
||||
domains = append(domains, sc.Text()) |
||||
} |
||||
|
||||
// Whatever happened, we're
|
||||
// done with the body now.
|
||||
body.Close() |
||||
|
||||
// Check if error reading body.
|
||||
if err := sc.Err(); err != nil { |
||||
return nil, gtserror.NewfAt(3, "error decoding into plain: %w", err) |
||||
} |
||||
|
||||
// Convert raw domains to permissions.
|
||||
perms := make([]gtsmodel.DomainPermission, 0, len(domains)) |
||||
for _, domainRaw := range domains { |
||||
|
||||
// Normalize + validate domain.
|
||||
domain, err := validateDomain(domainRaw) |
||||
if err != nil { |
||||
l.Warnf("skipping invalid domain %s: %+v", domainRaw, err) |
||||
continue |
||||
} |
||||
|
||||
// Instantiate the permission
|
||||
// as either block or allow.
|
||||
var perm gtsmodel.DomainPermission |
||||
switch permType { |
||||
case gtsmodel.DomainPermissionBlock: |
||||
perm = >smodel.DomainBlock{Domain: domain} |
||||
case gtsmodel.DomainPermissionAllow: |
||||
perm = >smodel.DomainAllow{Domain: domain} |
||||
} |
||||
|
||||
// We're done.
|
||||
perms = append(perms, perm) |
||||
} |
||||
|
||||
return perms, nil |
||||
} |
||||
|
||||
func validateDomain(domain string) (string, error) { |
||||
// Basic validation.
|
||||
if _, ok := dns.IsDomainName(domain); !ok { |
||||
err := fmt.Errorf("invalid domain name") |
||||
return "", err |
||||
} |
||||
|
||||
// Convert to punycode.
|
||||
domain, err := util.Punify(domain) |
||||
if err != nil { |
||||
err := fmt.Errorf("could not punify domain: %w", err) |
||||
return "", err |
||||
} |
||||
|
||||
// Check for invalid characters
|
||||
// after the punification process.
|
||||
if strings.ContainsAny(domain, "*, \n") { |
||||
err := fmt.Errorf("invalid char(s) in domain") |
||||
return "", err |
||||
} |
||||
|
||||
return domain, nil |
||||
} |
||||
|
||||
func (s *Subscriptions) existingCovered( |
||||
ctx context.Context, |
||||
permType gtsmodel.DomainPermissionType, |
||||
domain string, |
||||
higherPrios []*gtsmodel.DomainPermissionSubscription, |
||||
) ( |
||||
existingPerm gtsmodel.DomainPermission, |
||||
covered bool, |
||||
err error, |
||||
) { |
||||
// Check for existing perm
|
||||
// of appropriate type.
|
||||
var dbErr error |
||||
switch permType { |
||||
case gtsmodel.DomainPermissionBlock: |
||||
existingPerm, dbErr = s.state.DB.GetDomainBlock(ctx, domain) |
||||
case gtsmodel.DomainPermissionAllow: |
||||
existingPerm, dbErr = s.state.DB.GetDomainAllow(ctx, domain) |
||||
} |
||||
|
||||
if dbErr != nil && !errors.Is(dbErr, db.ErrNoEntries) { |
||||
// Real db error.
|
||||
err = dbErr |
||||
return |
||||
} |
||||
|
||||
if util.IsNil(existingPerm) { |
||||
// Can't be covered if
|
||||
// no existing perm.
|
||||
return |
||||
} |
||||
|
||||
subscriptionID := existingPerm.GetSubscriptionID() |
||||
if subscriptionID == "" { |
||||
// Can't be covered if
|
||||
// no subscription ID.
|
||||
return |
||||
} |
||||
|
||||
// Covered if subscription ID is in the slice
|
||||
// of higher-priority permission subscriptions.
|
||||
covered = slices.ContainsFunc( |
||||
higherPrios, |
||||
func(permSub *gtsmodel.DomainPermissionSubscription) bool { |
||||
return permSub.ID == subscriptionID |
||||
}, |
||||
) |
||||
|
||||
return |
||||
} |
||||
@ -0,0 +1,42 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package subscriptions |
||||
|
||||
import ( |
||||
"github.com/superseriousbusiness/gotosocial/internal/state" |
||||
"github.com/superseriousbusiness/gotosocial/internal/transport" |
||||
"github.com/superseriousbusiness/gotosocial/internal/typeutils" |
||||
) |
||||
|
||||
type Subscriptions struct { |
||||
state *state.State |
||||
transportController transport.Controller |
||||
tc *typeutils.Converter |
||||
} |
||||
|
||||
func New( |
||||
state *state.State, |
||||
transportController transport.Controller, |
||||
tc *typeutils.Converter, |
||||
) *Subscriptions { |
||||
return &Subscriptions{ |
||||
state: state, |
||||
transportController: transportController, |
||||
tc: tc, |
||||
} |
||||
} |
||||
@ -0,0 +1,538 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package subscriptions_test |
||||
|
||||
import ( |
||||
"context" |
||||
"errors" |
||||
"testing" |
||||
"time" |
||||
|
||||
"github.com/stretchr/testify/suite" |
||||
"github.com/superseriousbusiness/gotosocial/internal/db" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" |
||||
"github.com/superseriousbusiness/gotosocial/internal/subscriptions" |
||||
"github.com/superseriousbusiness/gotosocial/internal/util" |
||||
"github.com/superseriousbusiness/gotosocial/testrig" |
||||
) |
||||
|
||||
const ( |
||||
rMediaPath = "../../testrig/media" |
||||
rTemplatePath = "../../web/template" |
||||
) |
||||
|
||||
type SubscriptionsTestSuite struct { |
||||
suite.Suite |
||||
|
||||
testAccounts map[string]*gtsmodel.Account |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) SetupSuite() { |
||||
testrig.InitTestConfig() |
||||
testrig.InitTestLog() |
||||
suite.testAccounts = testrig.NewTestAccounts() |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) TestDomainBlocksCSV() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testStructs = testrig.SetupTestStructs(rMediaPath, rTemplatePath) |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
subscriptions = subscriptions.New( |
||||
testStructs.State, |
||||
testStructs.TransportController, |
||||
testStructs.TypeConverter, |
||||
) |
||||
|
||||
// Create a subscription for a CSV list of baddies.
|
||||
testSubscription = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/baddies.csv", |
||||
ContentType: gtsmodel.DomainPermSubContentTypeCSV, |
||||
} |
||||
) |
||||
defer testrig.TearDownTestStructs(testStructs) |
||||
|
||||
// Store test subscription.
|
||||
if err := testStructs.State.DB.PutDomainPermissionSubscription( |
||||
ctx, testSubscription, |
||||
); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Process all subscriptions.
|
||||
subscriptions.ProcessDomainPermissionSubscriptions(ctx, testSubscription.PermissionType) |
||||
|
||||
// We should now have blocks for
|
||||
// each domain on the subscribed list.
|
||||
for _, domain := range []string{ |
||||
"bumfaces.net", |
||||
"peepee.poopoo", |
||||
"nothanks.com", |
||||
} { |
||||
var ( |
||||
perm gtsmodel.DomainPermission |
||||
err error |
||||
) |
||||
if !testrig.WaitFor(func() bool { |
||||
perm, err = testStructs.State.DB.GetDomainBlock(ctx, domain) |
||||
return err == nil |
||||
}) { |
||||
suite.FailNowf("", "timed out waiting for domain %s", domain) |
||||
} |
||||
|
||||
suite.Equal(testSubscription.ID, perm.GetSubscriptionID()) |
||||
} |
||||
|
||||
// The just-fetched perm sub should
|
||||
// have ETag and count etc set now.
|
||||
permSub, err := testStructs.State.DB.GetDomainPermissionSubscriptionByID( |
||||
ctx, testSubscription.ID, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Should have some perms now.
|
||||
count, err := testStructs.State.DB.CountDomainPermissionSubscriptionPerms(ctx, permSub.ID) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
suite.Equal("bigbums6969", permSub.ETag) |
||||
suite.EqualValues(3, count) |
||||
suite.WithinDuration(time.Now(), permSub.FetchedAt, 1*time.Minute) |
||||
suite.WithinDuration(time.Now(), permSub.SuccessfullyFetchedAt, 1*time.Minute) |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) TestDomainBlocksJSON() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testStructs = testrig.SetupTestStructs(rMediaPath, rTemplatePath) |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
subscriptions = subscriptions.New( |
||||
testStructs.State, |
||||
testStructs.TransportController, |
||||
testStructs.TypeConverter, |
||||
) |
||||
|
||||
// Create a subscription for a JSON list of baddies.
|
||||
testSubscription = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/baddies.json", |
||||
ContentType: gtsmodel.DomainPermSubContentTypeJSON, |
||||
} |
||||
) |
||||
defer testrig.TearDownTestStructs(testStructs) |
||||
|
||||
// Store test subscription.
|
||||
if err := testStructs.State.DB.PutDomainPermissionSubscription( |
||||
ctx, testSubscription, |
||||
); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Process all subscriptions.
|
||||
subscriptions.ProcessDomainPermissionSubscriptions(ctx, testSubscription.PermissionType) |
||||
|
||||
// We should now have blocks for
|
||||
// each domain on the subscribed list.
|
||||
for _, domain := range []string{ |
||||
"bumfaces.net", |
||||
"peepee.poopoo", |
||||
"nothanks.com", |
||||
} { |
||||
var ( |
||||
perm gtsmodel.DomainPermission |
||||
err error |
||||
) |
||||
if !testrig.WaitFor(func() bool { |
||||
perm, err = testStructs.State.DB.GetDomainBlock(ctx, domain) |
||||
return err == nil |
||||
}) { |
||||
suite.FailNowf("", "timed out waiting for domain %s", domain) |
||||
} |
||||
|
||||
suite.Equal(testSubscription.ID, perm.GetSubscriptionID()) |
||||
} |
||||
|
||||
// The just-fetched perm sub should
|
||||
// have ETag and count etc set now.
|
||||
permSub, err := testStructs.State.DB.GetDomainPermissionSubscriptionByID( |
||||
ctx, testSubscription.ID, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Should have some perms now.
|
||||
count, err := testStructs.State.DB.CountDomainPermissionSubscriptionPerms(ctx, permSub.ID) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
suite.Equal("don't modify me daddy", permSub.ETag) |
||||
suite.EqualValues(3, count) |
||||
suite.WithinDuration(time.Now(), permSub.FetchedAt, 1*time.Minute) |
||||
suite.WithinDuration(time.Now(), permSub.SuccessfullyFetchedAt, 1*time.Minute) |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) TestDomainBlocksPlain() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testStructs = testrig.SetupTestStructs(rMediaPath, rTemplatePath) |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
subscriptions = subscriptions.New( |
||||
testStructs.State, |
||||
testStructs.TransportController, |
||||
testStructs.TypeConverter, |
||||
) |
||||
|
||||
// Create a subscription for a plain list of baddies.
|
||||
testSubscription = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/baddies.txt", |
||||
ContentType: gtsmodel.DomainPermSubContentTypePlain, |
||||
} |
||||
) |
||||
defer testrig.TearDownTestStructs(testStructs) |
||||
|
||||
// Store test subscription.
|
||||
if err := testStructs.State.DB.PutDomainPermissionSubscription( |
||||
ctx, testSubscription, |
||||
); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Process all subscriptions.
|
||||
subscriptions.ProcessDomainPermissionSubscriptions(ctx, testSubscription.PermissionType) |
||||
|
||||
// We should now have blocks for
|
||||
// each domain on the subscribed list.
|
||||
for _, domain := range []string{ |
||||
"bumfaces.net", |
||||
"peepee.poopoo", |
||||
"nothanks.com", |
||||
} { |
||||
var ( |
||||
perm gtsmodel.DomainPermission |
||||
err error |
||||
) |
||||
if !testrig.WaitFor(func() bool { |
||||
perm, err = testStructs.State.DB.GetDomainBlock(ctx, domain) |
||||
return err == nil |
||||
}) { |
||||
suite.FailNowf("", "timed out waiting for domain %s", domain) |
||||
} |
||||
|
||||
suite.Equal(testSubscription.ID, perm.GetSubscriptionID()) |
||||
} |
||||
|
||||
// The just-fetched perm sub should
|
||||
// have ETag and count etc set now.
|
||||
permSub, err := testStructs.State.DB.GetDomainPermissionSubscriptionByID( |
||||
ctx, testSubscription.ID, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Should have some perms now.
|
||||
count, err := testStructs.State.DB.CountDomainPermissionSubscriptionPerms(ctx, permSub.ID) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
suite.Equal("this is a legit etag i swear", permSub.ETag) |
||||
suite.EqualValues(3, count) |
||||
suite.WithinDuration(time.Now(), permSub.FetchedAt, 1*time.Minute) |
||||
suite.WithinDuration(time.Now(), permSub.SuccessfullyFetchedAt, 1*time.Minute) |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) TestDomainBlocksCSVETag() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testStructs = testrig.SetupTestStructs(rMediaPath, rTemplatePath) |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
subscriptions = subscriptions.New( |
||||
testStructs.State, |
||||
testStructs.TransportController, |
||||
testStructs.TypeConverter, |
||||
) |
||||
|
||||
// Create a subscription for a CSV list of baddies.
|
||||
// Include the ETag so it gets sent with the request.
|
||||
testSubscription = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/baddies.csv", |
||||
ContentType: gtsmodel.DomainPermSubContentTypeCSV, |
||||
ETag: "bigbums6969", |
||||
} |
||||
) |
||||
defer testrig.TearDownTestStructs(testStructs) |
||||
|
||||
// Store test subscription.
|
||||
if err := testStructs.State.DB.PutDomainPermissionSubscription( |
||||
ctx, testSubscription, |
||||
); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Process all subscriptions.
|
||||
subscriptions.ProcessDomainPermissionSubscriptions(ctx, testSubscription.PermissionType) |
||||
|
||||
// We should now NOT have blocks for the domains
|
||||
// on the list, as the remote will have returned
|
||||
// 304, indicating we should do nothing.
|
||||
for _, domain := range []string{ |
||||
"bumfaces.net", |
||||
"peepee.poopoo", |
||||
"nothanks.com", |
||||
} { |
||||
_, err := testStructs.State.DB.GetDomainBlock(ctx, domain) |
||||
if !errors.Is(err, db.ErrNoEntries) { |
||||
suite.FailNowf("", "domain perm %s created when it shouldn't be") |
||||
} |
||||
} |
||||
|
||||
// The just-fetched perm sub should
|
||||
// have ETag and count etc set now.
|
||||
permSub, err := testStructs.State.DB.GetDomainPermissionSubscriptionByID( |
||||
ctx, testSubscription.ID, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Should have no perms.
|
||||
count, err := testStructs.State.DB.CountDomainPermissionSubscriptionPerms(ctx, permSub.ID) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
suite.Equal("bigbums6969", permSub.ETag) |
||||
suite.Zero(count) |
||||
suite.WithinDuration(time.Now(), permSub.FetchedAt, 1*time.Minute) |
||||
suite.WithinDuration(time.Now(), permSub.SuccessfullyFetchedAt, 1*time.Minute) |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) TestDomainBlocks404() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testStructs = testrig.SetupTestStructs(rMediaPath, rTemplatePath) |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
subscriptions = subscriptions.New( |
||||
testStructs.State, |
||||
testStructs.TransportController, |
||||
testStructs.TypeConverter, |
||||
) |
||||
|
||||
// Create a subscription for a CSV list of baddies.
|
||||
// The endpoint will return a 404 so we can test erroring.
|
||||
testSubscription = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/does_not_exist.csv", |
||||
ContentType: gtsmodel.DomainPermSubContentTypeCSV, |
||||
} |
||||
) |
||||
defer testrig.TearDownTestStructs(testStructs) |
||||
|
||||
// Store test subscription.
|
||||
if err := testStructs.State.DB.PutDomainPermissionSubscription( |
||||
ctx, testSubscription, |
||||
); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Process all subscriptions.
|
||||
subscriptions.ProcessDomainPermissionSubscriptions(ctx, testSubscription.PermissionType) |
||||
|
||||
// The just-fetched perm sub should have an error set on it.
|
||||
permSub, err := testStructs.State.DB.GetDomainPermissionSubscriptionByID( |
||||
ctx, testSubscription.ID, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Should have no perms.
|
||||
count, err := testStructs.State.DB.CountDomainPermissionSubscriptionPerms(ctx, permSub.ID) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
suite.Zero(count) |
||||
suite.WithinDuration(time.Now(), permSub.FetchedAt, 1*time.Minute) |
||||
suite.Zero(permSub.SuccessfullyFetchedAt) |
||||
suite.Equal(`DereferenceDomainPermissions: GET request to https://lists.example.org/does_not_exist.csv failed: status="" body="{"error":"not found"}"`, permSub.Error) |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) TestDomainBlocksWrongContentTypeCSV() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testStructs = testrig.SetupTestStructs(rMediaPath, rTemplatePath) |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
subscriptions = subscriptions.New( |
||||
testStructs.State, |
||||
testStructs.TransportController, |
||||
testStructs.TypeConverter, |
||||
) |
||||
|
||||
// Create a subscription for a plaintext list of baddies,
|
||||
// but try to parse as CSV content type (shouldn't work).
|
||||
testSubscription = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/baddies.txt", |
||||
ContentType: gtsmodel.DomainPermSubContentTypeCSV, |
||||
} |
||||
) |
||||
defer testrig.TearDownTestStructs(testStructs) |
||||
|
||||
// Store test subscription.
|
||||
if err := testStructs.State.DB.PutDomainPermissionSubscription( |
||||
ctx, testSubscription, |
||||
); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Process all subscriptions.
|
||||
subscriptions.ProcessDomainPermissionSubscriptions(ctx, testSubscription.PermissionType) |
||||
|
||||
// The just-fetched perm sub should have an error set on it.
|
||||
permSub, err := testStructs.State.DB.GetDomainPermissionSubscriptionByID( |
||||
ctx, testSubscription.ID, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Should have no perms.
|
||||
count, err := testStructs.State.DB.CountDomainPermissionSubscriptionPerms(ctx, permSub.ID) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
suite.Zero(count) |
||||
suite.WithinDuration(time.Now(), permSub.FetchedAt, 1*time.Minute) |
||||
suite.Zero(permSub.SuccessfullyFetchedAt) |
||||
suite.Equal(`ProcessDomainPermissionSubscription: unexpected column headers in csv: [bumfaces.net]`, permSub.Error) |
||||
} |
||||
|
||||
func (suite *SubscriptionsTestSuite) TestDomainBlocksWrongContentTypePlain() { |
||||
var ( |
||||
ctx = context.Background() |
||||
testStructs = testrig.SetupTestStructs(rMediaPath, rTemplatePath) |
||||
testAccount = suite.testAccounts["admin_account"] |
||||
subscriptions = subscriptions.New( |
||||
testStructs.State, |
||||
testStructs.TransportController, |
||||
testStructs.TypeConverter, |
||||
) |
||||
|
||||
// Create a subscription for a plaintext list of baddies,
|
||||
// but try to parse as CSV content type (shouldn't work).
|
||||
testSubscription = >smodel.DomainPermissionSubscription{ |
||||
ID: "01JGE681TQSBPAV59GZXPKE62H", |
||||
Priority: 255, |
||||
Title: "whatever!", |
||||
PermissionType: gtsmodel.DomainPermissionBlock, |
||||
AsDraft: util.Ptr(false), |
||||
AdoptOrphans: util.Ptr(true), |
||||
CreatedByAccountID: testAccount.ID, |
||||
CreatedByAccount: testAccount, |
||||
URI: "https://lists.example.org/baddies.csv", |
||||
ContentType: gtsmodel.DomainPermSubContentTypePlain, |
||||
} |
||||
) |
||||
defer testrig.TearDownTestStructs(testStructs) |
||||
|
||||
// Store test subscription.
|
||||
if err := testStructs.State.DB.PutDomainPermissionSubscription( |
||||
ctx, testSubscription, |
||||
); err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Process all subscriptions.
|
||||
subscriptions.ProcessDomainPermissionSubscriptions(ctx, testSubscription.PermissionType) |
||||
|
||||
// The just-fetched perm sub should have an error set on it.
|
||||
permSub, err := testStructs.State.DB.GetDomainPermissionSubscriptionByID( |
||||
ctx, testSubscription.ID, |
||||
) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
// Should have no perms.
|
||||
count, err := testStructs.State.DB.CountDomainPermissionSubscriptionPerms(ctx, permSub.ID) |
||||
if err != nil { |
||||
suite.FailNow(err.Error()) |
||||
} |
||||
|
||||
suite.Zero(count) |
||||
suite.WithinDuration(time.Now(), permSub.FetchedAt, 1*time.Minute) |
||||
suite.Zero(permSub.SuccessfullyFetchedAt) |
||||
suite.Equal(`fetch successful but parsed zero usable results`, permSub.Error) |
||||
} |
||||
|
||||
func TestSubscriptionTestSuite(t *testing.T) { |
||||
suite.Run(t, new(SubscriptionsTestSuite)) |
||||
} |
||||
@ -0,0 +1,121 @@
|
||||
// GoToSocial
|
||||
// Copyright (C) GoToSocial Authors admin@gotosocial.org
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
//
|
||||
// This program is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU Affero General Public License for more details.
|
||||
//
|
||||
// You should have received a copy of the GNU Affero General Public License
|
||||
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
package transport |
||||
|
||||
import ( |
||||
"context" |
||||
"io" |
||||
"net/http" |
||||
|
||||
"github.com/superseriousbusiness/gotosocial/internal/gtserror" |
||||
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" |
||||
) |
||||
|
||||
type DereferenceDomainPermissionsResp struct { |
||||
// Set only if response was 200 OK.
|
||||
// It's up to the caller to close
|
||||
// this when they're done with it.
|
||||
Body io.ReadCloser |
||||
|
||||
// True if response
|
||||
// was 304 Not Modified.
|
||||
Unmodified bool |
||||
|
||||
// May be set
|
||||
// if 200 or 304.
|
||||
ETag string |
||||
} |
||||
|
||||
func (t *transport) DereferenceDomainPermissions( |
||||
ctx context.Context, |
||||
permSub *gtsmodel.DomainPermissionSubscription, |
||||
skipCache bool, |
||||
) (*DereferenceDomainPermissionsResp, error) { |
||||
// Prepare new HTTP request to endpoint
|
||||
req, err := http.NewRequestWithContext(ctx, "GET", permSub.URI, nil) |
||||
if err != nil { |
||||
return nil, err |
||||
} |
||||
|
||||
// Set basic auth header if necessary.
|
||||
if permSub.FetchUsername != "" || permSub.FetchPassword != "" { |
||||
req.SetBasicAuth(permSub.FetchUsername, permSub.FetchPassword) |
||||
} |
||||
|
||||
// Set relevant Accept headers.
|
||||
// Allow fallback in case target doesn't
|
||||
// negotiate content type correctly.
|
||||
req.Header.Add("Accept-Charset", "utf-8") |
||||
req.Header.Add("Accept", permSub.ContentType.String()+","+"*/*") |
||||
|
||||
// If skipCache is true, we want to skip setting Cache
|
||||
// headers so that we definitely don't get a 304 back.
|
||||
if !skipCache { |
||||
// If we've successfully fetched this list
|
||||
// before, set If-Modified-Since to last
|
||||
// success to make the request conditional.
|
||||
//
|
||||
// See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
|
||||
if !permSub.SuccessfullyFetchedAt.IsZero() { |
||||
timeStr := permSub.SuccessfullyFetchedAt.Format(http.TimeFormat) |
||||
req.Header.Add("If-Modified-Since", timeStr) |
||||
} |
||||
|
||||
// If we've got an ETag stored for this list, set
|
||||
// If-None-Match to make the request conditional.
|
||||
// See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag#caching_of_unchanged_resources.
|
||||
if len(permSub.ETag) != 0 { |
||||
req.Header.Add("If-None-Match", permSub.ETag) |
||||
} |
||||
} |
||||
|
||||
// Perform the HTTP request
|
||||
rsp, err := t.GET(req) |
||||
if err != nil { |
||||
return nil, err |
||||
} |
||||
|
||||
// If we have an unexpected / error response,
|
||||
// wrap + return as error. This will also drain
|
||||
// and close the response body for us.
|
||||
if rsp.StatusCode != http.StatusOK && |
||||
rsp.StatusCode != http.StatusNotModified { |
||||
err := gtserror.NewFromResponse(rsp) |
||||
return nil, err |
||||
} |
||||
|
||||
// Check already if we were given an ETag
|
||||
// we can use, as ETag is often returned
|
||||
// even on 304 Not Modified responses.
|
||||
permsResp := &DereferenceDomainPermissionsResp{ |
||||
ETag: rsp.Header.Get("Etag"), |
||||
} |
||||
|
||||
if rsp.StatusCode == http.StatusNotModified { |
||||
// Nothing has changed on the remote side
|
||||
// since we last fetched, so there's nothing
|
||||
// to do and we don't need to read the body.
|
||||
rsp.Body.Close() |
||||
permsResp.Unmodified = true |
||||
} else { |
||||
// Return the live body to the caller.
|
||||
permsResp.Body = rsp.Body |
||||
} |
||||
|
||||
return permsResp, nil |
||||
} |
||||
Loading…
Reference in new issue