johanels/penpot
1
0
Fork 0
mirror of https://github.com/penpot/penpot.git synced 2025-12-12 06:24:17 +01:00
Code Issues Actions 9 Packages Projects Releases Wiki Activity

🎉 Add management RPC API (#7700)
Some checks failed
_STAGING / build-bundle (push) Has been cancelled
Details
_STAGING / build-docker (push) Has been cancelled
Details
_DEVELOP / build-bundle (push) Has been cancelled
Details
_DEVELOP / build-docker (push) Has been cancelled
Details
Commit Message Check / Check Commit Message (push) Has been cancelled
Details

Browse Source 
* 🎉 Add management RPC API

And refactor internal http auth flow

* 📎 Adjust final url namings

* 📚 Update changelog
This commit is contained in:
Andrey Antukh
2025-11-10 17:10:59 +01:00
committed by GitHub
parent 1b50c13c4d
commit 28cf67e7ff
29 changed files with 782 additions and 321 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
CHANGES.md
View File

@@ -4,6 +4,12 @@
### :boom: Breaking changes & Deprecations ### :boom: Breaking changes & Deprecations
- The backend RPC API URLS are changed from `/api/rpc/command/<name>`
to `/api/main/methods/<name>` (the previou PATH is preserved for
backward compatibility; however, if you are a user of this API, it
is strongly recommended that you adapt your code to use the new
PATH.
### :rocket: Epics and highlights ### :rocket: Epics and highlights
### :heart: Community contributions (Thank you!) ### :heart: Community contributions (Thank you!)

1
backend/dev/user.clj
View File

@@ -27,6 +27,7 @@
[app.common.transit :as t] [app.common.transit :as t]
[app.common.types.file :as ctf] [app.common.types.file :as ctf]
[app.common.uuid :as uuid] [app.common.uuid :as uuid]
[app.common.uri :as u]
[app.config :as cf] [app.config :as cf]
[app.db :as db] [app.db :as db]
[app.main :as main] [app.main :as main]

23
backend/resources/app/templates/api-doc.tmpl
View File

@@ -4,7 +4,7 @@
<meta charset="utf-8" /> <meta charset="utf-8" />
<meta name="robots" content="noindex,nofollow"> <meta name="robots" content="noindex,nofollow">
<meta http-equiv="x-ua-compatible" content="ie=edge" /> <meta http-equiv="x-ua-compatible" content="ie=edge" />
<title>Builtin API Documentation - Penpot</title> <title>{{label|upper}} API Documentation</title>
<link rel="preconnect" href="https://fonts.googleapis.com"> <link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
@@ -19,7 +19,7 @@
<body> <body>
<main> <main>
<header> <header>
<h1>Penpot API Documentation (v{{version}})</h1> <h1>{{label|upper}}: API Documentation (v{{version}})</h1>
<small class="menu"> <small class="menu">
[ [
<nav> <nav>
@@ -31,9 +31,10 @@
</header> </header>
<section class="doc-content"> <section class="doc-content">
<h2>INTRODUCTION</h2> <h2>INTRODUCTION</h2>
<p>This documentation is intended to be a general overview of the penpot RPC API. <p>This documentation is intended to be a general overview of
If you prefer, you can use <a href="/api/openapi.json">OpenAPI</a> the {{label}} API. If you prefer, you can
and/or <a href="/api/openapi">SwaggerUI</a> as alternative.</p> use <a href="{{openapi}}">Swagger/OpenAPI</a> as
alternative.</p>
<h2>GENERAL NOTES</h2> <h2>GENERAL NOTES</h2>
@@ -43,7 +44,7 @@
that starts with <b>get-</b> in the name, can use GET HTTP that starts with <b>get-</b> in the name, can use GET HTTP
method which in many cases benefits from the HTTP cache.</p> method which in many cases benefits from the HTTP cache.</p>
{% block auth-section %}
<h3>Authentication</h3> <h3>Authentication</h3>
<p>The penpot backend right now offers two way for authenticate the request: <p>The penpot backend right now offers two way for authenticate the request:
<b>cookies</b> (the same mechanism that we use ourselves on accessing the API from the <b>cookies</b> (the same mechanism that we use ourselves on accessing the API from the
@@ -56,9 +57,10 @@
<p>The access token can be obtained on the appropriate section on profile settings <p>The access token can be obtained on the appropriate section on profile settings
and it should be provided using <b>`Authorization`</b> header with <b>`Token and it should be provided using <b>`Authorization`</b> header with <b>`Token
&lt;token-string&gt;`</b> value.</p> &lt;token-string&gt;`</b> value.</p>
{% endblock %}
<h3>Content Negotiation</h3> <h3>Content Negotiation</h3>
<p>The penpot API by default operates indistinctly with: <b>`application/json`</b> <p>This API operates indistinctly with: <b>`application/json`</b>
and <b>`application/transit+json`</b> content types. You should specify the and <b>`application/transit+json`</b> content types. You should specify the
desired content-type on the <b>`Accept`</b> header, the transit encoding is used desired content-type on the <b>`Accept`</b> header, the transit encoding is used
by default.</p> by default.</p>
@@ -75,13 +77,16 @@
standard <a href="https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API">Fetch standard <a href="https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API">Fetch
API</a></p> API</a></p>
{% block limits-section %}
<h3>Limits</h3> <h3>Limits</h3>
<p>The rate limit work per user basis (this means that different api keys share <p>The rate limit work per user basis (this means that different api keys share
the same rate limit). For now the limits are not documented because we are the same rate limit). For now the limits are not documented because we are
studying and analyzing the data. As a general rule, it should not be abused, if an studying and analyzing the data. As a general rule, it should not be abused, if an
abusive use is detected, we will proceed to block the user's access to the abusive use is detected, we will proceed to block the user's access to the
API.</p> API.</p>
{% endblock %}
{% block webhooks-section %}
<h3>Webhooks</h3> <h3>Webhooks</h3>
<p>All methods that emit webhook events are marked with flag <b>WEBHOOK</b>, the <p>All methods that emit webhook events are marked with flag <b>WEBHOOK</b>, the
data structure defined on each method represents the <i>payload</i> of the data structure defined on each method represents the <i>payload</i> of the
@@ -97,9 +102,11 @@
"profileId": "db601c95-045f-808b-8002-361312e63531" "profileId": "db601c95-045f-808b-8002-361312e63531"
} }
</pre> </pre>
{% endblock %}
</section> </section>
<section class="rpc-doc-content"> <section class="rpc-doc-content">
<h2>RPC METHODS REFERENCE:</h2> <h2>METHODS REFERENCE:</h2>
<ul class="rpc-items"> <ul class="rpc-items">
{% for item in methods %} {% for item in methods %}
{% include "app/templates/api-doc-entry.tmpl" with item=item %} {% include "app/templates/api-doc-entry.tmpl" with item=item %}

1
backend/resources/app/templates/main-api-doc.tmpl Normal file
View File

@@ -0,0 +1 @@
{% extends "app/templates/api-doc.tmpl" %}

10
backend/resources/app/templates/management-api-doc.tmpl Normal file
View File

@@ -0,0 +1,10 @@
{% extends "app/templates/api-doc.tmpl" %}
{% block auth-section %}
{% endblock %}
{% block limits-section %}
{% endblock %}
{% block webhooks-section %}
{% endblock %}

4
backend/resources/app/templates/openapi.tmpl
View File

@@ -7,7 +7,7 @@
name="description" name="description"
content="SwaggerUI" content="SwaggerUI"
/> />
<title>PENPOT Swagger UI</title> <title>{{label|upper}} API</title>
<style>{{swagger-css|safe}}</style> <style>{{swagger-css|safe}}</style>
</head> </head>
<body> <body>
@@ -16,7 +16,7 @@
<script> <script>
window.onload = () => { window.onload = () => {
window.ui = SwaggerUIBundle({ window.ui = SwaggerUIBundle({
url: '{{public-uri}}/api/openapi.json', url: '{{uri}}',
dom_id: '#swagger-ui', dom_id: '#swagger-ui',
presets: [ presets: [
SwaggerUIBundle.presets.apis, SwaggerUIBundle.presets.apis,

5
backend/src/app/auth/oidc.clj
View File

@@ -21,6 +21,8 @@
[app.email.whitelist :as email.whitelist] [app.email.whitelist :as email.whitelist]
[app.http.client :as http] [app.http.client :as http]
[app.http.errors :as errors] [app.http.errors :as errors]
[app.http.middleware :as mw]
[app.http.security :as sec]
[app.http.session :as session] [app.http.session :as session]
[app.loggers.audit :as audit] [app.loggers.audit :as audit]
[app.rpc :as rpc] [app.rpc :as rpc]
@@ -690,7 +692,8 @@
(defmethod ig/init-key ::routes (defmethod ig/init-key ::routes
[_ cfg] [_ cfg]
(let [cfg (update cfg :providers d/without-nils)] (let [cfg (update cfg :providers d/without-nils)]
["" {:middleware [[session/authz cfg] ["/api" {:middleware [[mw/cors]
[sec/client-header-check]
[provider-lookup cfg]]} [provider-lookup cfg]]}
["/auth/oauth" ["/auth/oauth"
["/:provider" ["/:provider"

5
backend/src/app/config.clj
View File

@@ -47,6 +47,7 @@
:auto-file-snapshot-timeout "3h" :auto-file-snapshot-timeout "3h"
:public-uri "http://localhost:3449" :public-uri "http://localhost:3449"
:host "localhost" :host "localhost"
:tenant "default" :tenant "default"
@@ -57,6 +58,8 @@
:objects-storage-backend "fs" :objects-storage-backend "fs"
:objects-storage-fs-directory "assets" :objects-storage-fs-directory "assets"
:auth-token-cookie-name "auth-token"
:assets-path "/internal/assets/" :assets-path "/internal/assets/"
:smtp-default-reply-to "Penpot <no-reply@example.com>" :smtp-default-reply-to "Penpot <no-reply@example.com>"
:smtp-default-from "Penpot <no-reply@example.com>" :smtp-default-from "Penpot <no-reply@example.com>"
@@ -90,7 +93,7 @@
[:secret-key {:optional true} :string] [:secret-key {:optional true} :string]
[:tenant {:optional false} :string] [:tenant {:optional false} :string]
[:public-uri {:optional false} :string] [:public-uri {:optional false} ::sm/uri]
[:host {:optional false} :string] [:host {:optional false} :string]
[:http-server-port {:optional true} ::sm/int] [:http-server-port {:optional true} ::sm/int]

13
backend/src/app/http.clj
View File

@@ -25,7 +25,6 @@
[app.main :as-alias main] [app.main :as-alias main]
[app.metrics :as mtx] [app.metrics :as mtx]
[app.rpc :as-alias rpc] [app.rpc :as-alias rpc]
[app.rpc.doc :as-alias rpc.doc]
[app.setup :as-alias setup] [app.setup :as-alias setup]
[integrant.core :as ig] [integrant.core :as ig]
[reitit.core :as r] [reitit.core :as r]
@@ -149,7 +148,6 @@
[:map [:map
[::ws/routes schema:routes] [::ws/routes schema:routes]
[::rpc/routes schema:routes] [::rpc/routes schema:routes]
[::rpc.doc/routes schema:routes]
[::oidc/routes schema:routes] [::oidc/routes schema:routes]
[::assets/routes schema:routes] [::assets/routes schema:routes]
[::debug/routes schema:routes] [::debug/routes schema:routes]
@@ -171,8 +169,9 @@
[sec/sec-fetch-metadata] [sec/sec-fetch-metadata]
[mw/params] [mw/params]
[mw/format-response] [mw/format-response]
[session/soft-auth cfg] [mw/auth {:bearer (partial session/decode-token cfg)
[actoken/soft-auth cfg] :cookie (partial session/decode-token cfg)
:token (partial actoken/decode-token cfg)}]
[mw/parse-request] [mw/parse-request]
[mw/errors errors/handle] [mw/errors errors/handle]
[mw/restrict-methods]]} [mw/restrict-methods]]}
@@ -188,9 +187,5 @@
(::mgmt/routes cfg)] (::mgmt/routes cfg)]
(::ws/routes cfg) (::ws/routes cfg)
["/api" {:middleware [[mw/cors]
[sec/client-header-check]]}
(::oidc/routes cfg) (::oidc/routes cfg)
(::rpc.doc/routes cfg) (::rpc/routes cfg)]]))
(::rpc/routes cfg)]]]))

60
backend/src/app/http/access_token.clj
View File

@@ -9,23 +9,19 @@
[app.common.logging :as l] [app.common.logging :as l]
[app.config :as cf] [app.config :as cf]
[app.db :as db] [app.db :as db]
[app.http.auth :as-alias http.auth]
[app.main :as-alias main] [app.main :as-alias main]
[app.setup :as-alias setup] [app.setup :as-alias setup]
[app.tokens :as tokens] [app.tokens :as tokens]))
[yetti.request :as yreq]))
(def header-re #"(?i)^Token\s+(.*)") (defn decode-token
(defn get-token
[request]
(some->> (yreq/get-header request "authorization")
(re-matches header-re)
(second)))
(defn- decode-token
[cfg token] [cfg token]
(when token (try
(tokens/verify cfg {:token token :iss "access-token"}))) (tokens/verify cfg {:token token :iss "access-token"})
(catch Throwable cause
(l/trc :hint "exception on decoding token"
:token token
:cause cause))))
(def sql:get-token-data (def sql:get-token-data
"SELECT perms, profile_id, expires_at "SELECT perms, profile_id, expires_at
@@ -35,47 +31,27 @@
OR (expires_at > now()));") OR (expires_at > now()));")
(defn- get-token-data (defn- get-token-data
[pool token-id] [pool claims]
(when-not (db/read-only? pool) (when-not (db/read-only? pool)
(when-let [token-id (-> (deref claims) (get :tid))]
(some-> (db/exec-one! pool [sql:get-token-data token-id]) (some-> (db/exec-one! pool [sql:get-token-data token-id])
(update :perms db/decode-pgarray #{})))) (update :perms db/decode-pgarray #{})))))
(defn- wrap-soft-auth
"Soft Authentication, will be executed synchronously on the undertow
worker thread."
[handler cfg]
(letfn [(handle-request [request]
(try
(let [token (get-token request)
claims (decode-token cfg token)]
(cond-> request
(map? claims)
(assoc ::id (:tid claims))))
(catch Throwable cause
(l/trace :hint "exception on decoding malformed token" :cause cause)
request)))]
(fn [request]
(handler (handle-request request)))))
(defn- wrap-authz (defn- wrap-authz
"Authorization middleware, will be executed synchronously on vthread."
[handler {:keys [::db/pool]}] [handler {:keys [::db/pool]}]
(fn [request] (fn [{:keys [::http.auth/token-type] :as request}]
(let [{:keys [perms profile-id expires-at]} (some->> (::id request) (get-token-data pool))] (if (= :token token-type)
(let [{:keys [perms profile-id expires-at]} (some->> (get request ::http.auth/claims)
(get-token-data pool))]
(handler (cond-> request (handler (cond-> request
(some? perms) (some? perms)
(assoc ::perms perms) (assoc ::perms perms)
(some? profile-id) (some? profile-id)
(assoc ::profile-id profile-id) (assoc ::profile-id profile-id)
(some? expires-at) (some? expires-at)
(assoc ::expires-at expires-at)))))) (assoc ::expires-at expires-at))))
(def soft-auth (handler request))))
{:name ::soft-auth
:compile (fn [& _]
(when (contains? cf/flags :access-tokens)
wrap-soft-auth))})
(def authz (def authz
{:name ::authz {:name ::authz

7
backend/src/app/http/errors.clj
View File

@@ -13,6 +13,7 @@
[app.config :as cf] [app.config :as cf]
[app.http :as-alias http] [app.http :as-alias http]
[app.http.access-token :as-alias actoken] [app.http.access-token :as-alias actoken]
[app.http.auth :as-alias auth]
[app.http.session :as-alias session] [app.http.session :as-alias session]
[app.util.inet :as inet] [app.util.inet :as inet]
[clojure.spec.alpha :as s] [clojure.spec.alpha :as s]
@@ -22,16 +23,14 @@
(defn request->context (defn request->context
"Extracts error report relevant context data from request." "Extracts error report relevant context data from request."
[request] [request]
(let [claims (-> {} (let [claims (some-> (get request ::auth/claims) deref)]
(into (::session/token-claims request))
(into (::actoken/token-claims request)))]
(-> (cf/logging-context) (-> (cf/logging-context)
(assoc :request/path (:path request)) (assoc :request/path (:path request))
(assoc :request/method (:method request)) (assoc :request/method (:method request))
(assoc :request/params (:params request)) (assoc :request/params (:params request))
(assoc :request/user-agent (yreq/get-header request "user-agent")) (assoc :request/user-agent (yreq/get-header request "user-agent"))
(assoc :request/ip-addr (inet/parse-request request)) (assoc :request/ip-addr (inet/parse-request request))
(assoc :request/profile-id (:uid claims)) (assoc :request/profile-id (get claims :uid))
(assoc :version/frontend (or (yreq/get-header request "x-frontend-version") "unknown"))))) (assoc :version/frontend (or (yreq/get-header request "x-frontend-version") "unknown")))))
(defmulti handle-error (defmulti handle-error

18
backend/src/app/http/management.clj
View File

@@ -13,7 +13,7 @@
[app.common.time :as ct] [app.common.time :as ct]
[app.config :as cf] [app.config :as cf]
[app.db :as db] [app.db :as db]
[app.http.access-token :refer [get-token]] [app.http.middleware :as mw]
[app.main :as-alias main] [app.main :as-alias main]
[app.rpc.commands.profile :as cmd.profile] [app.rpc.commands.profile :as cmd.profile]
[app.setup :as-alias setup] [app.setup :as-alias setup]
@@ -32,20 +32,6 @@
[_ params] [_ params]
(assert (db/pool? (::db/pool params)) "expect valid database pool")) (assert (db/pool? (::db/pool params)) "expect valid database pool"))
(def ^:private auth
{:name ::auth
:compile
(fn [_ _]
(fn [handler shared-key]
(if shared-key
(fn [request]
(let [token (get-token request)]
(if (= token shared-key)
(handler request)
{::yres/status 403})))
(fn [_ _]
{::yres/status 403}))))})
(def ^:private default-system (def ^:private default-system
{:name ::default-system {:name ::default-system
:compile :compile
@@ -65,7 +51,7 @@
(defmethod ig/init-key ::routes (defmethod ig/init-key ::routes
[_ cfg] [_ cfg]
["" {:middleware [[auth (cf/get :management-api-shared-key)] ["" {:middleware [[mw/shared-key-auth (cf/get :management-api-shared-key)]
[default-system cfg] [default-system cfg]
[transaction]]} [transaction]]}
["/authenticate" ["/authenticate"

59
backend/src/app/http/middleware.clj
View File

@@ -12,6 +12,7 @@
[app.common.schema :as-alias sm] [app.common.schema :as-alias sm]
[app.common.transit :as t] [app.common.transit :as t]
[app.config :as cf] [app.config :as cf]
[app.http.auth :as-alias auth]
[app.http.errors :as errors] [app.http.errors :as errors]
[app.util.pointer-map :as pmap] [app.util.pointer-map :as pmap]
[cuerdas.core :as str] [cuerdas.core :as str]
@@ -240,3 +241,61 @@
(if (contains? allowed method) (if (contains? allowed method)
(handler request) (handler request)
{::yres/status 405}))))))}) {::yres/status 405}))))))})
(defn- wrap-auth
[handler decoders]
(let [token-re
#"(?i)^(Token|Bearer)\s+(.*)"
get-token-from-authorization
(fn [request]
(when-let [[_ token-type token] (some->> (yreq/get-header request "authorization")
(re-matches token-re))]
(if (= "token" (str/lower token-type))
[:token token]
[:bearer token])))
get-token-from-cookie
(fn [request]
(let [cname (cf/get :auth-token-cookie-name)
token (some-> (yreq/get-cookie request cname) :value)]
(when-not (str/empty? token)
[:cookie token])))
get-token
(some-fn get-token-from-cookie get-token-from-authorization)
process-request
(fn [request]
(if-let [[token-type token] (get-token request)]
(let [request (-> request
(assoc ::auth/token token)
(assoc ::auth/token-type token-type))
decoder (get decoders token-type)]
(if (fn? decoder)
(assoc request ::auth/claims (delay (decoder token)))
request))
request))]
(fn [request]
(-> request process-request handler))))
(def auth
{:name ::auth
:compile (constantly wrap-auth)})
(defn- wrap-shared-key-auth
[handler shared-key]
(if shared-key
(fn [request]
(let [key (yreq/get-header request "x-shared-key")]
(if (= key shared-key)
(handler request)
{::yres/status 403})))
(fn [_ _]
{::yres/status 403})))
(def shared-key-auth
{:name ::shared-key-auth
:compile (constantly wrap-shared-key-auth)})

83
backend/src/app/http/session.clj
View File

@@ -14,6 +14,7 @@
[app.config :as cf] [app.config :as cf]
[app.db :as db] [app.db :as db]
[app.db.sql :as sql] [app.db.sql :as sql]
[app.http.auth :as-alias http.auth]
[app.http.session.tasks :as-alias tasks] [app.http.session.tasks :as-alias tasks]
[app.main :as-alias main] [app.main :as-alias main]
[app.setup :as-alias setup] [app.setup :as-alias setup]
@@ -26,13 +27,6 @@
;; DEFAULTS ;; DEFAULTS
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; A default cookie name for storing the session.
(def default-auth-token-cookie-name "auth-token")
;; A cookie that we can use to check from other sites of the same
;; domain if a user is authenticated.
(def default-auth-data-cookie-name "auth-data")
;; Default value for cookie max-age ;; Default value for cookie max-age
(def default-cookie-max-age (ct/duration {:days 7})) (def default-cookie-max-age (ct/duration {:days 7}))
@@ -169,7 +163,7 @@
[{:keys [::manager]}] [{:keys [::manager]}]
(assert (manager? manager) "expected valid session manager") (assert (manager? manager) "expected valid session manager")
(fn [request response] (fn [request response]
(let [cname (cf/get :auth-token-cookie-name default-auth-token-cookie-name) (let [cname (cf/get :auth-token-cookie-name)
cookie (yreq/get-cookie request cname)] cookie (yreq/get-cookie request cname)]
(l/trc :hint "delete" :profile-id (:profile-id request)) (l/trc :hint "delete" :profile-id (:profile-id request))
(some->> (:value cookie) (delete! manager)) (some->> (:value cookie) (delete! manager))
@@ -183,21 +177,14 @@
(tokens/generate cfg {:iss "authentication" (tokens/generate cfg {:iss "authentication"
:iat created-at :iat created-at
:uid profile-id})) :uid profile-id}))
(defn- decode-token (defn decode-token
[cfg token] [cfg token]
(when token (try
(tokens/verify cfg {:token token :iss "authentication"}))) (tokens/verify cfg {:token token :iss "authentication"})
(catch Throwable cause
(defn- get-token (l/trc :hint "exception on decoding token"
[request] :token token
(let [cname (cf/get :auth-token-cookie-name default-auth-token-cookie-name) :cause cause))))
cookie (some-> (yreq/get-cookie request cname) :value)]
(when-not (str/empty? cookie)
cookie)))
(defn- get-session
[manager token]
(some->> token (read manager)))
(defn- renew-session? (defn- renew-session?
[{:keys [updated-at] :as session}] [{:keys [updated-at] :as session}]
@@ -205,44 +192,38 @@
(let [elapsed (ct/diff updated-at (ct/now))] (let [elapsed (ct/diff updated-at (ct/now))]
(neg? (compare default-renewal-max-age elapsed))))) (neg? (compare default-renewal-max-age elapsed)))))
(defn- wrap-soft-auth (defn- wrap-authz
[handler {:keys [::manager] :as cfg}] [handler {:keys [::manager] :as cfg}]
(assert (manager? manager) "expected valid session manager") (assert (manager? manager) "expected valid session manager")
(letfn [(handle-request [request] (fn [{:keys [::http.auth/token-type] :as request}]
(try (cond
(let [token (get-token request) (= token-type :cookie)
claims (decode-token cfg token)] (let [session (some->> (get request ::http.auth/token)
(cond-> request (read manager))
(map? claims)
(-> (assoc ::token-claims claims)
(assoc ::token token))))
(catch Throwable cause
(l/trc :hint "exception on decoding malformed token" :cause cause)
request)))]
(fn [request]
(handler (handle-request request)))))
(defn- wrap-authz
[handler {:keys [::manager]}]
(assert (manager? manager) "expected valid session manager")
(fn [request]
(let [session (get-session manager (::token request))
request (cond-> request request (cond-> request
(some? session) (some? session)
(assoc ::profile-id (:profile-id session) (-> (assoc ::profile-id (:profile-id session))
::id (:id session))) (assoc ::id (:id session))))
response (handler request)] response (handler request)]
(if (renew-session? session) (if (renew-session? session)
(let [session (update! manager session)] (let [session (update! manager session)]
(-> response (-> response
(assign-auth-token-cookie session))) (assign-auth-token-cookie session)))
response)))) response))
(def soft-auth (= token-type :bearer)
{:name ::soft-auth (let [session (some->> (get request ::http.auth/token)
:compile (constantly wrap-soft-auth)}) (read manager))
request (cond-> request
(some? session)
(-> (assoc ::profile-id (:profile-id session))
(assoc ::id (:id session))))]
(handler request))
:else
(handler request))))
(def authz (def authz
{:name ::authz {:name ::authz
@@ -259,7 +240,7 @@
secure? (contains? cf/flags :secure-session-cookies) secure? (contains? cf/flags :secure-session-cookies)
strict? (contains? cf/flags :strict-session-cookies) strict? (contains? cf/flags :strict-session-cookies)
cors? (contains? cf/flags :cors) cors? (contains? cf/flags :cors)
name (cf/get :auth-token-cookie-name default-auth-token-cookie-name) name (cf/get :auth-token-cookie-name)
comment (str "Renewal at: " (ct/format-inst renewal :rfc1123)) comment (str "Renewal at: " (ct/format-inst renewal :rfc1123))
cookie {:path "/" cookie {:path "/"
:http-only true :http-only true
@@ -272,7 +253,7 @@
(defn- clear-auth-token-cookie (defn- clear-auth-token-cookie
[response] [response]
(let [cname (cf/get :auth-token-cookie-name default-auth-token-cookie-name)] (let [cname (cf/get :auth-token-cookie-name)]
(update response :cookies assoc cname {:path "/" :value "" :max-age 0}))) (update response :cookies assoc cname {:path "/" :value "" :max-age 0})))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

20
backend/src/app/main.clj
View File

@@ -21,7 +21,7 @@
[app.http.client :as-alias http.client] [app.http.client :as-alias http.client]
[app.http.debug :as-alias http.debug] [app.http.debug :as-alias http.debug]
[app.http.management :as mgmt] [app.http.management :as mgmt]
[app.http.session :as-alias session] [app.http.session :as session]
[app.http.session.tasks :as-alias session.tasks] [app.http.session.tasks :as-alias session.tasks]
[app.http.websocket :as http.ws] [app.http.websocket :as http.ws]
[app.loggers.webhooks :as-alias webhooks] [app.loggers.webhooks :as-alias webhooks]
@@ -31,7 +31,6 @@
[app.redis :as-alias rds] [app.redis :as-alias rds]
[app.rpc :as-alias rpc] [app.rpc :as-alias rpc]
[app.rpc.climit :as-alias climit] [app.rpc.climit :as-alias climit]
[app.rpc.doc :as-alias rpc.doc]
[app.setup :as-alias setup] [app.setup :as-alias setup]
[app.srepl :as-alias srepl] [app.srepl :as-alias srepl]
[app.storage :as-alias sto] [app.storage :as-alias sto]
@@ -280,7 +279,6 @@
{::session/manager (ig/ref ::session/manager) {::session/manager (ig/ref ::session/manager)
::db/pool (ig/ref ::db/pool) ::db/pool (ig/ref ::db/pool)
::rpc/routes (ig/ref ::rpc/routes) ::rpc/routes (ig/ref ::rpc/routes)
::rpc.doc/routes (ig/ref ::rpc.doc/routes)
::setup/props (ig/ref ::setup/props) ::setup/props (ig/ref ::setup/props)
::mtx/routes (ig/ref ::mtx/routes) ::mtx/routes (ig/ref ::mtx/routes)
::oidc/routes (ig/ref ::oidc/routes) ::oidc/routes (ig/ref ::oidc/routes)
@@ -337,11 +335,23 @@
::email/blacklist (ig/ref ::email/blacklist) ::email/blacklist (ig/ref ::email/blacklist)
::email/whitelist (ig/ref ::email/whitelist)} ::email/whitelist (ig/ref ::email/whitelist)}
:app.rpc.doc/routes :app.rpc/management-methods
{:app.rpc/methods (ig/ref :app.rpc/methods)} {::http.client/client (ig/ref ::http.client/client)
::db/pool (ig/ref ::db/pool)
::rds/pool (ig/ref ::rds/pool)
::wrk/executor (ig/ref ::wrk/netty-executor)
::session/manager (ig/ref ::session/manager)
::sto/storage (ig/ref ::sto/storage)
::mtx/metrics (ig/ref ::mtx/metrics)
::mbus/msgbus (ig/ref ::mbus/msgbus)
::rds/client (ig/ref ::rds/client)
::setup/props (ig/ref ::setup/props)}
::rpc/routes ::rpc/routes
{::rpc/methods (ig/ref :app.rpc/methods) {::rpc/methods (ig/ref :app.rpc/methods)
::rpc/management-methods (ig/ref :app.rpc/management-methods)
;; FIXME: revisit if db/pool is necessary here
::db/pool (ig/ref ::db/pool) ::db/pool (ig/ref ::db/pool)
::session/manager (ig/ref ::session/manager) ::session/manager (ig/ref ::session/manager)
::setup/props (ig/ref ::setup/props)} ::setup/props (ig/ref ::setup/props)}

141
backend/src/app/rpc.clj
View File

@@ -13,11 +13,14 @@
[app.common.schema :as sm] [app.common.schema :as sm]
[app.common.spec :as us] [app.common.spec :as us]
[app.common.time :as ct] [app.common.time :as ct]
[app.common.uri :as u]
[app.config :as cf] [app.config :as cf]
[app.db :as db] [app.db :as db]
[app.http :as-alias http] [app.http :as-alias http]
[app.http.access-token :as actoken] [app.http.access-token :as actoken]
[app.http.client :as-alias http.client] [app.http.client :as-alias http.client]
[app.http.middleware :as mw]
[app.http.security :as sec]
[app.http.session :as session] [app.http.session :as session]
[app.loggers.audit :as audit] [app.loggers.audit :as audit]
[app.main :as-alias main] [app.main :as-alias main]
@@ -26,6 +29,7 @@
[app.redis :as rds] [app.redis :as rds]
[app.rpc.climit :as climit] [app.rpc.climit :as climit]
[app.rpc.cond :as cond] [app.rpc.cond :as cond]
[app.rpc.doc :as doc]
[app.rpc.helpers :as rph] [app.rpc.helpers :as rph]
[app.rpc.retry :as retry] [app.rpc.retry :as retry]
[app.rpc.rlimit :as rlimit] [app.rpc.rlimit :as rlimit]
@@ -36,7 +40,6 @@
[clojure.spec.alpha :as s] [clojure.spec.alpha :as s]
[cuerdas.core :as str] [cuerdas.core :as str]
[integrant.core :as ig] [integrant.core :as ig]
[promesa.core :as p]
[yetti.request :as yreq] [yetti.request :as yreq]
[yetti.response :as yres])) [yetti.response :as yres]))
@@ -44,7 +47,7 @@
(defn- default-handler (defn- default-handler
[_] [_]
(p/rejected (ex/error :type :not-found))) (ex/raise :type :not-found))
(defn- handle-response-transformation (defn- handle-response-transformation
[response request mdata] [response request mdata]
@@ -92,10 +95,12 @@
(str/blank? origin)) (str/blank? origin))
origin))) origin)))
(defn- rpc-handler (defn- make-rpc-handler
"Ring handler that dispatches cmd requests and convert between "Ring handler that dispatches cmd requests and convert between
internal async flow into ring async flow." internal async flow into ring async flow."
[methods {:keys [params path-params method] :as request}] [methods]
(let [methods (update-vals methods peek)]
(fn [{:keys [params path-params method] :as request}]
(let [handler-name (:type path-params) (let [handler-name (:type path-params)
etag (yreq/get-header request "if-none-match") etag (yreq/get-header request "if-none-match")
profile-id (or (::session/profile-id request) profile-id (or (::session/profile-id request)
@@ -126,9 +131,10 @@
:code :method-not-allowed :code :method-not-allowed
:hint "method not allowed for this request")) :hint "method not allowed for this request"))
;; FIXME: why we have this cond enabled here, we need to move it outside this handler
(binding [cond/*enabled* true] (binding [cond/*enabled* true]
(let [response (handler-fn data)] (let [response (handler-fn data)]
(handle-response request response))))) (handle-response request response)))))))
(defn- wrap-metrics (defn- wrap-metrics
"Wrap service method with metrics measurement." "Wrap service method with metrics measurement."
@@ -205,7 +211,7 @@
::sm/explain (explain params))))))) ::sm/explain (explain params)))))))
f)) f))
(defn- wrap-all (defn- wrap
[cfg f mdata] [cfg f mdata]
(as-> f $ (as-> f $
(wrap-db-transaction cfg $ mdata) (wrap-db-transaction cfg $ mdata)
@@ -219,17 +225,30 @@
(wrap-params-validation cfg $ mdata) (wrap-params-validation cfg $ mdata)
(wrap-authentication cfg $ mdata))) (wrap-authentication cfg $ mdata)))
(defn- wrap (defn- wrap-management
[cfg f mdata] [cfg f mdata]
(l/trc :hint "register method" :name (::sv/name mdata)) (as-> f $
(let [f (wrap-all cfg f mdata)] (wrap-db-transaction cfg $ mdata)
(partial f cfg))) (retry/wrap-retry cfg $ mdata)
(climit/wrap cfg $ mdata)
(wrap-metrics cfg $ mdata)
(wrap-audit cfg $ mdata)
(wrap-spec-conform cfg $ mdata)
(wrap-params-validation cfg $ mdata)
(wrap-authentication cfg $ mdata)))
(defn- process-method (defn- process-method
[cfg [vfn mdata]] [cfg module wrap-fn [f mdata]]
[(keyword (::sv/name mdata)) [mdata (wrap cfg vfn mdata)]]) (l/trc :hint "add method" :module module :name (::sv/name mdata))
(let [f (wrap-fn cfg f mdata)
k (keyword (::sv/name mdata))]
[k [mdata (partial f cfg)]]))
(defn- resolve-command-methods ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; API METHODS
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn- resolve-methods
[cfg] [cfg]
(let [cfg (assoc cfg ::type "command" ::metrics-id :rpc-command-timing)] (let [cfg (assoc cfg ::type "command" ::metrics-id :rpc-command-timing)]
(->> (sv/scan-ns (->> (sv/scan-ns
@@ -258,7 +277,7 @@
'app.rpc.commands.verify-token 'app.rpc.commands.verify-token
'app.rpc.commands.viewer 'app.rpc.commands.viewer
'app.rpc.commands.webhooks) 'app.rpc.commands.webhooks)
(map (partial process-method cfg)) (map (partial process-method cfg "rpc" wrap))
(into {})))) (into {}))))
(def ^:private schema:methods-params (def ^:private schema:methods-params
@@ -282,7 +301,49 @@
(defmethod ig/init-key ::methods (defmethod ig/init-key ::methods
[_ cfg] [_ cfg]
(let [cfg (d/without-nils cfg)] (let [cfg (d/without-nils cfg)]
(resolve-command-methods cfg))) (resolve-methods cfg)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; MANAGEMENT METHODS
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn- resolve-management-methods
[cfg]
(let [cfg (assoc cfg ::type "management" ::metrics-id :rpc-management-timing)]
(->> (sv/scan-ns
'app.rpc.management.subscription)
(map (partial process-method cfg "management" wrap-management))
(into {}))))
(def ^:private schema:management-methods-params
[:map {:title "management-methods-params"}
::session/manager
::http.client/client
::db/pool
::rds/pool
::mbus/msgbus
::sto/storage
::mtx/metrics
::setup/props])
(defmethod ig/assert-key ::management-methods
[_ params]
(assert (sm/check schema:management-methods-params params)))
(defmethod ig/init-key ::management-methods
[_ cfg]
(let [cfg (d/without-nils cfg)]
(resolve-management-methods cfg)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; ROUTES
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn- redirect
[href]
(fn [_]
{::yres/status 308
::yres/headers {"location" (str href)}}))
(def ^:private schema:methods (def ^:private schema:methods
[:map-of :keyword [:tuple :map ::sm/fn]]) [:map-of :keyword [:tuple :map ::sm/fn]])
@@ -297,11 +358,49 @@
(assert (db/pool? (::db/pool params)) "expect valid database pool") (assert (db/pool? (::db/pool params)) "expect valid database pool")
(assert (some? (::setup/props params))) (assert (some? (::setup/props params)))
(assert (session/manager? (::session/manager params)) "expect valid session manager") (assert (session/manager? (::session/manager params)) "expect valid session manager")
(assert (valid-methods? (::methods params)) "expect valid methods map")) (assert (valid-methods? (::methods params)) "expect valid methods map")
(assert (valid-methods? (::management-methods params)) "expect valid methods map"))
(defmethod ig/init-key ::routes (defmethod ig/init-key ::routes
[_ {:keys [::methods] :as cfg}] [_ {:keys [::methods ::management-methods] :as cfg}]
(let [methods (update-vals methods peek)]
[["/rpc" {:middleware [[session/authz cfg] (let [public-uri (cf/get :public-uri)]
[actoken/authz cfg]]} ["/api"
["/command/:type" {:handler (partial rpc-handler methods)}]]]))
["/management"
["/methods/:type"
{:middleware [[mw/shared-key-auth (cf/get :management-api-shared-key)]
[session/authz cfg]]
:handler (make-rpc-handler management-methods)}]
(doc/routes :methods management-methods
:label "management"
:base-uri (u/join public-uri "/api/management")
:description "MANAGEMENT API")]
["/main"
["/methods/:type"
{:middleware [[mw/cors]
[sec/client-header-check]
[session/authz cfg]
[actoken/authz cfg]]
:handler (make-rpc-handler methods)}]
(doc/routes :methods methods
:label "main"
:base-uri (u/join public-uri "/api/main")
:description "MAIN API")]
;; BACKWARD COMPATIBILITY
["/_doc" {:handler (redirect (u/join public-uri "/api/main/doc"))}]
["/doc" {:handler (redirect (u/join public-uri "/api/main/doc"))}]
["/openapi" {:handler (redirect (u/join public-uri "/api/main/doc/openapi"))}]
["/openapi.join" {:handler (redirect (u/join public-uri "/api/main/doc/openapi.json"))}]
["/rpc/command/:type"
{:middleware [[mw/cors]
[sec/client-header-check]
[session/authz cfg]
[actoken/authz cfg]]
:handler (make-rpc-handler methods)}]]))

1
backend/src/app/rpc/commands/access_token.clj
View File

@@ -28,6 +28,7 @@
expires-at (some-> expiration (ct/in-future)) expires-at (some-> expiration (ct/in-future))
created-at (ct/now) created-at (ct/now)
token (tokens/generate cfg {:iss "access-token" token (tokens/generate cfg {:iss "access-token"
:uid profile-id
:iat created-at :iat created-at
:tid token-id}) :tid token-id})

110
backend/src/app/rpc/doc.clj
View File

@@ -16,6 +16,7 @@
[app.common.schema.desc-native :as smdn] [app.common.schema.desc-native :as smdn]
[app.common.schema.openapi :as oapi] [app.common.schema.openapi :as oapi]
[app.common.schema.registry :as sr] [app.common.schema.registry :as sr]
[app.common.uri :as u]
[app.config :as cf] [app.config :as cf]
[app.http.sse :as-alias sse] [app.http.sse :as-alias sse]
[app.loggers.webhooks :as-alias webhooks] [app.loggers.webhooks :as-alias webhooks]
@@ -25,7 +26,6 @@
[clojure.java.io :as io] [clojure.java.io :as io]
[clojure.spec.alpha :as s] [clojure.spec.alpha :as s]
[cuerdas.core :as str] [cuerdas.core :as str]
[integrant.core :as ig]
[pretty-spec.core :as ps] [pretty-spec.core :as ps]
[yetti.response :as-alias yres])) [yetti.response :as-alias yres]))
@@ -33,8 +33,8 @@
;; DOC (human readable) ;; DOC (human readable)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn- prepare-doc-context (defn- context
[methods] [{:keys [methods entrypoint label openapi]}]
(letfn [(fmt-spec [mdata] (letfn [(fmt-spec [mdata]
(when-let [spec (ex/ignoring (s/spec (::sv/spec mdata)))] (when-let [spec (ex/ignoring (s/spec (::sv/spec mdata)))]
(with-out-str (with-out-str
@@ -62,8 +62,10 @@
:added (::added mdata) :added (::added mdata)
:changes (some->> (::changes mdata) (partition-all 2) (map vec)) :changes (some->> (::changes mdata) (partition-all 2) (map vec))
:spec (fmt-spec mdata) :spec (fmt-spec mdata)
:entrypoint (str (cf/get :public-uri) "/api/rpc/command/" (::sv/name mdata)) :entrypoint (-> entrypoint
(u/ensure-path-slash)
(u/join (::sv/name mdata))
(str))
:params-schema-js (fmt-schema :js mdata ::sm/params) :params-schema-js (fmt-schema :js mdata ::sm/params)
:result-schema-js (fmt-schema :js mdata ::sm/result) :result-schema-js (fmt-schema :js mdata ::sm/result)
:webhook-schema-js (fmt-schema :js mdata ::sm/webhook) :webhook-schema-js (fmt-schema :js mdata ::sm/webhook)
@@ -72,6 +74,9 @@
:webhook-schema-clj (fmt-schema :clj mdata ::sm/webhook)})] :webhook-schema-clj (fmt-schema :clj mdata ::sm/webhook)})]
{:version (:main cf/version) {:version (:main cf/version)
:label label
:entrypoint (str entrypoint)
:openapi (str openapi)
:methods :methods
(->> methods (->> methods
(map val) (map val)
@@ -80,17 +85,19 @@
(map get-context) (map get-context)
(sort-by (juxt :module :name)))})) (sort-by (juxt :module :name)))}))
(defn- doc-handler (defn- handler
[context] [& {:keys [template] :as options}]
(if (contains? cf/flags :backend-api-doc) (if (contains? cf/flags :backend-api-doc)
(let [context (delay (context options))
template (or template "app/templates/api-doc.tmpl")]
(fn [request] (fn [request]
(let [params (:query-params request) (let [params (:query-params request)
pstyle (:type params "js") pstyle (:type params "js")
context (assoc @context :param-style pstyle)] context (assoc @context :param-style pstyle)]
{::yres/status 200 {::yres/status 200
::yres/body (-> (io/resource "app/templates/api-doc.tmpl") ::yres/body (-> (io/resource template)
(tmpl/render context))})) (tmpl/render context))})))
(fn [_] (fn [_]
{::yres/status 404}))) {::yres/status 404})))
@@ -98,8 +105,8 @@
;; OPENAPI / SWAGGER (v3.1) ;; OPENAPI / SWAGGER (v3.1)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn prepare-openapi-context (defn- openapi-context
[methods] [{:keys [methods entrypoint description]}]
(let [definitions (atom {}) (let [definitions (atom {})
options {:registry sr/default-registry options {:registry sr/default-registry
::oapi/definitions-path "#/components/schemas/" ::oapi/definitions-path "#/components/schemas/"
@@ -112,7 +119,9 @@
(fn [tsx schema] (fn [tsx schema]
(let [schema (sm/schema schema) (let [schema (sm/schema schema)
example (sm/generate schema) example (sm/generate schema)
example (sm/encode schema example output-transformer)] example (sm/encode schema example output-transformer)
example (json/encode example :key-fn json/write-camel-key)]
{:default {:default
{:description "A default response" {:description "A default response"
:content :content
@@ -123,7 +132,9 @@
gen-params-doc gen-params-doc
(fn [tsx schema] (fn [tsx schema]
(let [example (sm/generate schema) (let [example (sm/generate schema)
example (sm/encode schema example output-transformer)] example (sm/encode schema example output-transformer)
example (json/encode example :key-fn json/write-camel-key)]
{:required true {:required true
:content :content
{"application/json" {"application/json"
@@ -158,34 +169,35 @@
(map gen-method-doc) (map gen-method-doc)
(sort-by (juxt :module :name)) (sort-by (juxt :module :name))
(map (fn [doc] (map (fn [doc]
[(str/ffmt "/command/%" (:name doc)) (:repr doc)])) [(:name doc) (:repr doc)]))
(into {})))] (into {})))]
{:openapi "3.0.0" {:openapi "3.0.0"
:info {:version (:main cf/version)} :info {:version (:main cf/version)}
:servers [{:url (str/ffmt "%/api/rpc" (cf/get :public-uri)) :servers [{:url (str entrypoint)
;; :description "penpot backend" :description (or description "")}]
}]
:paths paths :paths paths
:components {:schemas @definitions}})) :components {:schemas @definitions}}))
(defn openapi-json-handler (defn- openapi-json-handler
[context] [& {:as options}]
(if (contains? cf/flags :backend-openapi-doc) (if (contains? cf/flags :backend-openapi-doc)
(let [context (delay (openapi-context options))]
(fn [_] (fn [_]
{::yres/status 200 {::yres/status 200
::yres/headers {"content-type" "application/json; charset=utf-8"} ::yres/headers {"content-type" "application/json; charset=utf-8"}
::yres/body (json/encode @context)}) ::yres/body (json/encode @context)}))
(fn [_] (fn [_]
{::yres/status 404}))) {::yres/status 404})))
(defn openapi-handler (defn- openapi-handler
[] [& {:keys [uri label]}]
(if (contains? cf/flags :backend-openapi-doc) (if (contains? cf/flags :backend-openapi-doc)
(fn [_] (fn [_]
(let [swagger-js (slurp (io/resource "app/assets/swagger-ui-4.18.3.js")) (let [swagger-js (slurp (io/resource "app/assets/swagger-ui-4.18.3.js"))
swagger-cs (slurp (io/resource "app/assets/swagger-ui-4.18.3.css")) swagger-cs (slurp (io/resource "app/assets/swagger-ui-4.18.3.css"))
context {:public-uri (cf/get :public-uri) context {:uri (str uri)
:label label
:swagger-js swagger-js :swagger-js swagger-js
:swagger-css swagger-cs}] :swagger-css swagger-cs}]
{::yres/status 200 {::yres/status 200
@@ -196,27 +208,43 @@
{::yres/status 404}))) {::yres/status 404})))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; MODULE INIT ;; ROUTES HELPER
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defmethod ig/assert-key ::routes (defn routes
[_ params] [& {:keys [label base-uri description methods]}]
(assert (sm/valid? ::rpc/methods (::rpc/methods params)) "expected valid methods")) (let [entrypoint
(-> base-uri
(u/ensure-path-slash)
(u/join "methods"))
openapi
(-> base-uri
(u/ensure-path-slash)
(u/join "doc/openapi"))
template
(case label
"management" "app/templates/management-api-doc.tmpl"
"main" "app/templates/main-api-doc.tmpl")]
(defmethod ig/init-key ::routes
[_ {:keys [::rpc/methods] :as cfg}]
[(let [context (delay (prepare-doc-context methods))]
[["/_doc"
{:handler (doc-handler context)
:allowed-methods #{:get}}]
["/doc" ["/doc"
{:handler (doc-handler context) ["" {:handler (handler :methods methods
:allowed-methods #{:get}}]]) :label label
:entrypoint entrypoint
(let [context (delay (prepare-openapi-context methods))] :openapi openapi
[["/openapi" :template template)
{:handler (openapi-handler)
:allowed-methods #{:get}}] :allowed-methods #{:get}}]
["/openapi"
{:handler (openapi-handler
:uri (u/join openapi "openapi.json")
:label label)
:allowed-methods #{:get}}]
["/openapi.json" ["/openapi.json"
{:handler (openapi-json-handler context) {:handler (openapi-json-handler {:entrypoint entrypoint
:allowed-methods #{:get}}]])]) :description description
:methods methods})
:allowed-methods #{:get}}]]))

183
backend/src/app/rpc/management/subscription.clj Normal file
View File

@@ -0,0 +1,183 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns app.rpc.management.subscription
(:require
[app.common.logging :as l]
[app.common.schema :as sm]
[app.common.schema.generators :as sg]
[app.common.time :as ct]
[app.db :as db]
[app.rpc :as-alias rpc]
[app.rpc.commands.profile :as profile]
[app.rpc.doc :as doc]
[app.util.services :as sv]))
;; ---- RPC METHOD: AUTHENTICATE
(def ^:private
schema:authenticate-params
[:map {:title "authenticate-params"}])
(def ^:private
schema:authenticate-result
[:map {:title "authenticate-result"}
[:profile-id ::sm/uuid]])
(sv/defmethod ::auth
{::doc/added "2.12"
::sm/params schema:authenticate-params
::sm/result schema:authenticate-result}
[_ {:keys [::rpc/profile-id]}]
{:profile-id profile-id})
;; ---- RPC METHOD: GET-CUSTOMER
;; FIXME: move to app.common.time
(def ^:private schema:timestamp
(sm/type-schema
{:type ::timestamp
:pred ct/inst?
:type-properties
{:title "inst"
:description "The same as :app.common.time/inst but encodes to epoch"
:error/message "should be an instant"
:gen/gen (->> (sg/small-int)
(sg/fmap (fn [v] (ct/inst v))))
:decode/string #(some-> % ct/inst)
:encode/string #(some-> % inst-ms)
:decode/json #(some-> % ct/inst)
:encode/json #(some-> % inst-ms)}}))
(def ^:private schema:subscription
[:map {:title "Subscription"}
[:id ::sm/text]
[:customer-id ::sm/text]
[:type [:enum
"unlimited"
"professional"
"enterprise"]]
[:status [:enum
"active"
"canceled"
"incomplete"
"incomplete_expired"
"past_due"
"paused"
"trialing"
"unpaid"]]
[:billing-period [:enum
"month"
"day"
"week"
"year"]]
[:quantity :int]
[:description [:maybe ::sm/text]]
[:created-at schema:timestamp]
[:start-date [:maybe schema:timestamp]]
[:ended-at [:maybe schema:timestamp]]
[:trial-end [:maybe schema:timestamp]]
[:trial-start [:maybe schema:timestamp]]
[:cancel-at [:maybe schema:timestamp]]
[:canceled-at [:maybe schema:timestamp]]
[:current-period-end [:maybe schema:timestamp]]
[:current-period-start [:maybe schema:timestamp]]
[:cancel-at-period-end :boolean]
[:cancellation-details
[:map {:title "CancellationDetails"}
[:comment [:maybe ::sm/text]]
[:reason [:maybe ::sm/text]]
[:feedback [:maybe
[:enum
"customer_service"
"low_quality"
"missing_feature"
"other"
"switched_service"
"too_complex"
"too_expensive"
"unused"]]]]]])
(def ^:private sql:get-customer-slots
"WITH teams AS (
SELECT tpr.team_id AS id,
tpr.profile_id AS profile_id
FROM team_profile_rel AS tpr
WHERE tpr.is_owner IS true
AND tpr.profile_id = ?
), teams_with_slots AS (
SELECT tpr.team_id AS id,
count(*) AS total
FROM team_profile_rel AS tpr
WHERE tpr.team_id IN (SELECT id FROM teams)
AND tpr.can_edit IS true
GROUP BY 1
ORDER BY 2
)
SELECT max(total) AS total FROM teams_with_slots;")
(defn- get-customer-slots
[cfg profile-id]
(let [result (db/exec-one! cfg [sql:get-customer-slots profile-id])]
(:total result)))
(def ^:private schema:get-customer-params
[:map])
(def ^:private schema:get-customer-result
[:map
[:id ::sm/uuid]
[:name :string]
[:num-editors ::sm/int]
[:subscription {:optional true} schema:subscription]])
(sv/defmethod ::get-customer
{::doc/added "2.12"
::sm/params schema:get-customer-params
::sm/result schema:get-customer-result}
[cfg {:keys [::rpc/profile-id]}]
(let [profile (profile/get-profile cfg profile-id)]
{:id (get profile :id)
:name (get profile :fullname)
:email (get profile :email)
:num-editors (get-customer-slots cfg profile-id)
:subscription (-> profile :props :subscription)}))
;; ---- RPC METHOD: GET-CUSTOMER
(def ^:private schema:update-customer-params
[:map
[:subscription [:maybe schema:subscription]]])
(def ^:private schema:update-customer-result
[:map])
(sv/defmethod ::update-customer
{::doc/added "2.12"
::sm/params schema:update-customer-params
::sm/result schema:update-customer-result}
[cfg {:keys [::rpc/profile-id subscription]}]
(let [{:keys [props] :as profile}
(profile/get-profile cfg profile-id ::db/for-update true)
props
(assoc props :subscription subscription)]
(l/dbg :hint "update customer"
:profile-id (str profile-id)
:subscription-type (get subscription :type)
:subscription-status (get subscription :status)
:subscription-quantity (get subscription :quantity))
(db/update! cfg :profile
{:props (db/tjson props)}
{:id profile-id}
{::db/return-keys false})
nil))

2
backend/src/app/util/template.clj
View File

@@ -9,7 +9,7 @@
[app.common.exceptions :as ex] [app.common.exceptions :as ex]
[selmer.parser :as sp])) [selmer.parser :as sp]))
;; (sp/cache-off!) (sp/cache-off!)
(defn render (defn render
[path context] [path context]

57
backend/test/backend_tests/http_middleware_access_token_test.clj
View File

@@ -1,57 +0,0 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns backend-tests.http-middleware-access-token-test
(:require
[app.db :as db]
[app.http.access-token]
[app.main :as-alias main]
[app.rpc :as-alias rpc]
[app.rpc.commands.access-token]
[app.tokens :as tokens]
[backend-tests.helpers :as th]
[clojure.test :as t]
[mockery.core :refer [with-mocks]]))
(t/use-fixtures :once th/state-init)
(t/use-fixtures :each th/database-reset)
(t/deftest soft-auth-middleware
(let [profile (th/create-profile* 1)
token (db/tx-run! th/*system* app.rpc.commands.access-token/create-access-token (:id profile) "test" nil)
request (volatile! nil)
handler (#'app.http.access-token/wrap-soft-auth
(fn [req] (vreset! request req))
th/*system*)]
(with-mocks [m1 {:target 'app.http.access-token/get-token
:return nil}]
(handler {})
(t/is (= {} @request)))
(with-mocks [m1 {:target 'app.http.access-token/get-token
:return (:token token)}]
(handler {})
(let [token-id (get @request :app.http.access-token/id)]
(t/is (= token-id (:id token)))))))
(t/deftest authz-middleware
(let [profile (th/create-profile* 1)
token (db/tx-run! th/*system* app.rpc.commands.access-token/create-access-token (:id profile) "test" nil)
request (volatile! {})
handler (#'app.http.access-token/wrap-authz
(fn [req] (vreset! request req))
th/*system*)]
(handler nil)
(t/is (nil? @request))
(handler {:app.http.access-token/id (:id token)})
(t/is (= #{} (:app.http.access-token/perms @request)))
(t/is (= (:id profile) (:app.http.access-token/profile-id @request)))))

161
backend/test/backend_tests/http_middleware_test.clj Normal file
View File

@@ -0,0 +1,161 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns backend-tests.http-middleware-test
(:require
[app.common.time :as ct]
[app.db :as db]
[app.http.access-token]
[app.http.auth :as-alias auth]
[app.http.middleware :as mw]
[app.http.session :as session]
[app.main :as-alias main]
[app.rpc :as-alias rpc]
[app.rpc.commands.access-token]
[app.tokens :as tokens]
[backend-tests.helpers :as th]
[clojure.test :as t]
[mockery.core :refer [with-mocks]]
[yetti.request :as yreq]
[yetti.response :as yres]))
(t/use-fixtures :once th/state-init)
(t/use-fixtures :each th/database-reset)
(defrecord DummyRequest [headers cookies]
yreq/IRequestCookies
(get-cookie [_ name]
{:value (get cookies name)})
yreq/IRequest
(get-header [_ name]
(get headers name)))
(t/deftest auth-middleware-1
(let [request (volatile! nil)
handler (#'app.http.middleware/wrap-auth
(fn [req] (vreset! request req))
{})]
(handler (->DummyRequest {} {}))
(t/is (nil? (::auth/token-type @request)))
(t/is (nil? (::auth/token @request)))
(handler (->DummyRequest {"authorization" "Token aaaa"} {}))
(t/is (= :token (::auth/token-type @request)))
(t/is (= "aaaa" (::auth/token @request)))))
(t/deftest auth-middleware-2
(let [request (volatile! nil)
handler (#'app.http.middleware/wrap-auth
(fn [req] (vreset! request req))
{})]
(handler (->DummyRequest {} {}))
(t/is (nil? (::auth/token-type @request)))
(t/is (nil? (::auth/token @request)))
(t/is (nil? (::auth/claims @request)))
(handler (->DummyRequest {"authorization" "Bearer aaaa"} {}))
(t/is (= :bearer (::auth/token-type @request)))
(t/is (= "aaaa" (::auth/token @request)))
(t/is (nil? (::auth/claims @request)))))
(t/deftest auth-middleware-3
(let [request (volatile! nil)
handler (#'app.http.middleware/wrap-auth
(fn [req] (vreset! request req))
{})]
(handler (->DummyRequest {} {}))
(t/is (nil? (::auth/token-type @request)))
(t/is (nil? (::auth/token @request)))
(t/is (nil? (::auth/claims @request)))
(handler (->DummyRequest {} {"auth-token" "foobar"}))
(t/is (= :cookie (::auth/token-type @request)))
(t/is (= "foobar" (::auth/token @request)))
(t/is (nil? (::auth/claims @request)))))
(t/deftest auth-middleware-4
(let [request (volatile! nil)
handler (#'app.http.middleware/wrap-auth
(fn [req] (vreset! request req))
{:cookie (fn [_] "foobaz")})]
(handler (->DummyRequest {} {}))
(t/is (nil? (::auth/token-type @request)))
(t/is (nil? (::auth/token @request)))
(t/is (nil? (::auth/claims @request)))
(handler (->DummyRequest {} {"auth-token" "foobar"}))
(t/is (= :cookie (::auth/token-type @request)))
(t/is (= "foobar" (::auth/token @request)))
(t/is (delay? (::auth/claims @request)))
(t/is (= "foobaz" (-> @request ::auth/claims deref)))))
(t/deftest shared-key-auth
(let [handler (#'app.http.middleware/wrap-shared-key-auth
(fn [req] {::yres/status 200})
"secret-key")]
(let [response (handler (->DummyRequest {} {}))]
(t/is (= 403 (::yres/status response))))
(let [response (handler (->DummyRequest {"x-shared-key" "secret-key2"} {}))]
(t/is (= 403 (::yres/status response))))
(let [response (handler (->DummyRequest {"x-shared-key" "secret-key"} {}))]
(t/is (= 200 (::yres/status response))))))
(t/deftest access-token-authz
(let [profile (th/create-profile* 1)
token (db/tx-run! th/*system* app.rpc.commands.access-token/create-access-token (:id profile) "test" nil)
request (volatile! {})
handler (#'app.http.access-token/wrap-authz
(fn [req] (vreset! request req))
th/*system*)]
(handler nil)
(t/is (nil? @request))
(handler {::auth/claims (delay {:tid (:id token)})
::auth/token-type :token})
(t/is (= #{} (:app.http.access-token/perms @request)))
(t/is (= (:id profile) (:app.http.access-token/profile-id @request)))))
(t/deftest session-authz
(let [manager (session/inmemory-manager)
profile (th/create-profile* 1)
handler (-> (fn [req] req)
(#'session/wrap-authz {::session/manager manager})
(#'mw/wrap-auth {}))]
(let [response (handler (->DummyRequest {} {"auth-token" "foobar"}))]
(t/is (= :cookie (::auth/token-type response)))
(t/is (= "foobar" (::auth/token response))))
(session/write! manager "foobar" {:profile-id (:id profile)
:user-agent "user agent"
:created-at (ct/now)})
(let [response (handler (->DummyRequest {} {"auth-token" "foobar"}))]
(t/is (= :cookie (::auth/token-type response)))
(t/is (= "foobar" (::auth/token response)))
(t/is (= (:id profile) (::session/profile-id response)))
(t/is (= "foobar" (::session/id response))))))

2
backend/test/backend_tests/rpc_doc_test.clj
View File

@@ -23,7 +23,7 @@
(smt/check! (smt/check!
(smt/for [context (->> sg/int (smt/for [context (->> sg/int
(sg/fmap (fn [_] (sg/fmap (fn [_]
(rpc.doc/prepare-openapi-context (::rpc/methods th/*system*)))))] (#'rpc.doc/openapi-context (::rpc/methods th/*system*)))))]
(try (try
(json/encode context) (json/encode context)
true true

9
common/src/app/common/uri.cljc
View File

@@ -8,6 +8,7 @@
(:refer-clojure :exclude [uri?]) (:refer-clojure :exclude [uri?])
(:require (:require
[app.common.data.macros :as dm] [app.common.data.macros :as dm]
[cuerdas.core :as str]
[lambdaisland.uri :as u] [lambdaisland.uri :as u]
[lambdaisland.uri.normalize :as un]) [lambdaisland.uri.normalize :as un])
#?(:clj #?(:clj
@@ -58,6 +59,14 @@
(map (fn [[k v]] [(key-fn k) (value-fn v)])))) (map (fn [[k v]] [(key-fn k) (value-fn v)]))))
(u/map->query-string)))) (u/map->query-string))))
(defn ensure-path-slash
[u]
(update (uri u) :path
(fn [path]
(if (str/ends-with? path "/")
path
(str path "/")))))
#?(:clj #?(:clj
(defmethod print-method lambdaisland.uri.URI [^URI this ^java.io.Writer writer] (defmethod print-method lambdaisland.uri.URI [^URI this ^java.io.Writer writer]
(.write writer "#") (.write writer "#")

2
frontend/playwright/ui/pages/BasePage.js
View File

@@ -9,7 +9,7 @@ export class BasePage {
); );
} }
const url = typeof path === "string" ? `**/api/rpc/command/${path}` : path; const url = typeof path === "string" ? `**/api/main/methods/${path}` : path;
const interceptConfig = { const interceptConfig = {
status: 200, status: 200,
contentType: "application/transit+json", contentType: "application/transit+json",

2
frontend/playwright/ui/specs/example.spec.js
View File

@@ -1,7 +1,7 @@
import { test, expect } from "@playwright/test"; import { test, expect } from "@playwright/test";
test("Has title", async ({ page }) => { test("Has title", async ({ page }) => {
await page.route("**/api/rpc/command/get-profile", (route) => { await page.route("**/api/main/methods/get-profile", (route) => {
route.fulfill({ route.fulfill({
status: 200, status: 200,
contentType: "application/transit+json", contentType: "application/transit+json",

2
frontend/src/app/main/data/event.cljs
View File

@@ -243,7 +243,7 @@
(defn- persist-events (defn- persist-events
[events] [events]
(if (seq events) (if (seq events)
(let [uri (u/join cf/public-uri "api/rpc/command/push-audit-events") (let [uri (u/join cf/public-uri "api/main/methods/push-audit-events")
params {:uri uri params {:uri uri
:method :post :method :post
:credentials "include" :credentials "include"

4
frontend/src/app/main/repo.cljs
View File

@@ -115,7 +115,7 @@
request request
{:method method {:method method
:uri (u/join cf/public-uri "api/rpc/command/" nid) :uri (u/join cf/public-uri "api/main/methods/" nid)
:credentials "include" :credentials "include"
:headers {"accept" "application/transit+json,text/event-stream,*/*" :headers {"accept" "application/transit+json,text/event-stream,*/*"
"x-external-session-id" (cf/external-session-id) "x-external-session-id" (cf/external-session-id)
@@ -207,7 +207,7 @@
(defmethod cmd! ::multipart-upload (defmethod cmd! ::multipart-upload
[id params] [id params]
(->> (http/send! {:method :post (->> (http/send! {:method :post
:uri (u/join cf/public-uri "api/rpc/command/" (name id)) :uri (u/join cf/public-uri "api/main/methods/" (name id))
:credentials "include" :credentials "include"
:headers {"x-external-session-id" (cf/external-session-id) :headers {"x-external-session-id" (cf/external-session-id)
"x-event-origin" (::ev/origin (meta params))} "x-event-origin" (::ev/origin (meta params))}

2
frontend/src/app/worker/thumbnails.cljs
View File

@@ -43,7 +43,7 @@
(defn- request-data-for-thumbnail (defn- request-data-for-thumbnail
[file-id revn] [file-id revn]
(let [path "api/rpc/command/get-file-data-for-thumbnail" (let [path "api/main/methods/get-file-data-for-thumbnail"
params {:file-id file-id params {:file-id file-id
:revn revn :revn revn
:strip-frames-with-thumbnails true} :strip-frames-with-thumbnails true}