Jekyll2021-02-26T16:40:55+00:00https://alinush.github.io//feed.xmlAlin TomescuTBD
Alin TomescuWhat is a Merkle tree?2021-02-25T00:00:00+00:002021-02-25T00:00:00+00:00https://alinush.github.io//2021/02/25/what-is-a-merkle-tree<p class="info"><strong>tl;dr:</strong> In this post, we demystify Merkle trees for beginners.
We give simple, illustrated explanations, we detail applications they are used in and reason about their security formally.</p>
<!-- more -->
<p>For more details, see this full post on <a href="https://decentralizedthoughts.github.io/2020-12-22-what-is-a-merkle-tree/">Decentralized Thoughts</a>.</p>Alin Tomescutl;dr: In this post, we demystify Merkle trees for beginners. We give simple, illustrated explanations, we detail applications they are used in and reason about their security formally. For more details, see this full post on Decentralized Thoughts.Authenticated Dictionaries with Cross-Incremental Proof (Dis)aggregation2020-11-26T00:00:00+00:002020-11-26T00:00:00+00:00https://alinush.github.io//2020/11/26/Authenticated-Dictionaries-with-Cross-Incremental-Proof-Disaggregation<p class="info"><strong>tl;dr:</strong> We build an authenticated dictionary (AD) from Catalano Fiore vector commitments that has constant-sized, aggregatable proofs and supports a stronger notion of cross-incremental proof disaggregation.
Our AD could be used for stateless validation in cryptocurrencies with smart contract execution.
In a future post, we will extend this AD with stronger security, non-membership proofs and append-only proofs, which makes it applicable to transparency logging.</p>
<p>This is joint work with my brilliant (ex-)labmates from MIT, <a href="https://twitter.com/SuperAluex">Alex (Yu) Xia</a> and <a href="https://github.com/znewman01">Zack Newman</a>.</p>
<p><strong>Authenticated dictionaries (ADs)</strong> are an important cryptographic primitive which lies at the core of cryptocurrencies such as <em>Ethereum</em> and of transparency logs such as <em>Certificate Transparency (CT)</em>.
Typically, ADs are constructed by Merkleizing a lexicographically-ordered data structure such as a binary search tree, a prefix tree or a skip list.
However, our work takes a different, more algebraic direction, building upon the <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html">Catalano-Fiore (CF) vector commitment (VC) scheme</a>.
This has the advantage of giving us <em>constant-sized</em> proofs which are updatable and aggregatable, with a novel notion of <strong>cross-incrementality</strong>.
Importantly, this combination of feautres is not supported by Merkle trees or any other previous VC scheme.</p>
<!--more-->
<p>In a nutshell, in this post, we:</p>
<ul>
<li>Extend CF with a larger index space to accommodate dictionary keys, obtaining an authenticated dictionary,</li>
<li>Extend our AD to support updating proofs and digests after removing keys from the dictionary,</li>
<li>Introduce a novel notion of <strong>cross-incremental proof (dis)aggregation</strong> w.r.t. different ADs</li>
</ul>
<p>In a future post, we will explain how we:</p>
<ul>
<li>Strengthen our AD’s security to handle more adversarial settings such as transparency logs,</li>
<li>Add proofs of non-membership,</li>
<li>Add append-only proofs.</li>
</ul>
<p>Our algebraic approach is not novel in itself and we relate to the previous line of work that explores building ADs from non-Merkle techniques in <strong>our full paper</strong><sup id="fnref:TXN20e" role="doc-noteref"><a href="#fn:TXN20e" class="footnote">1</a></sup>.
You can also see a quick comparison in our <a href="https://github.com/alinush/authdict-talk/raw/zkstudyclub/talk.pdf">zkStudyClub slides</a>.</p>
<h2 id="preliminaries">Preliminaries</h2>
<p hidden="">$$
\def\Adv{\mathcal{A}}
\def\Badv{\mathcal{B}}
\def\GenGho{\mathsf{GenGroup}_?}
\def\Ghosz{|\Gho|}
\def\Ghoid{1_{\Gho}}
\def\primes{\mathsf{Primes}}
\def\QRn{\mathsf{QR}_N}
\def\multirootexp{\mathsf{MultiRootExp}}
\def\rootfactor{\mathsf{RootFactor}}
\def\vect#1{\mathbf{#1}}
$$</p>
<p>We often use the following notation:</p>
<ul>
<li>$\lambda$ denotes the security parameter of our schemes</li>
<li>$[n] = \{1,2,\dots, n\}$</li>
<li>We denote a vector using a <strong>bolded</strong> variable $\vect{v} = [v_1, \dots, v_n]$</li>
<li>$\Gho$ denotes the hidden-order group our constructions use
<ul>
<li>e.g., \(\Gho = \ZNs =\{a \mathrel\vert \gcd(a,N) = 1\}\)</li>
</ul>
</li>
<li>Let $D$ be a dictionary over a set of keys $K$ that maps each key $k\in K$ to its value $v = D(k)$</li>
<li>We sometimes use $k\in D$ to indicate that key $k$ has some value in the dictionary</li>
<li>We sometimes use $(k,v)\in D$ notation to indicate that key $k$ has value $v$ in the dictionary</li>
<li>We sometimes use $D’ = D\setminus K$ to refer to the new dictionary $D’$ obtained after removing all keys in $K$ (and their values) from the dictionary $D$</li>
</ul>
<p>This post assumes knowledge of:</p>
<ul>
<li>Greatest common divisor (GCD) of two integers $x, y$ denoted by $\gcd(x,y)$</li>
<li>The Extended Euclidean Algorithm (EEA) for computing Bezout coefficients $x,y\in \Z$ such that $ax + by = \gcd(a,b)$</li>
<li><a href="/2020/11/24/RSA-Accumulators.html">RSA accumulators</a>
<ul>
<li>An RSA accumulator for a set \(T = \{b_1, \dots, b_n\}\) of elements where each $b_i$ can be hashed to a <em>prime representative</em> $e_i$ is \(a = g^{\prod_{i \in [n]} e_i}\).</li>
<li>An RSA <em>membership witness</em> for $b_i$ is just \(w_i = a^{1/e_i} = g^{\prod_{j\in[n], j\ne i} e_j}\).</li>
<li>To verify it, just check $w_i^{e_i} = a$.</li>
<li>Recall all RSA membership witnesses can be computed using an algorithm by <em>Sander et al.</em><sup id="fnref:SSY01" role="doc-noteref"><a href="#fn:SSY01" class="footnote">2</a></sup> baptised as \(\rootfactor\) by <em>Boneh et al.</em><sup id="fnref:BBF18" role="doc-noteref"><a href="#fn:BBF18" class="footnote">3</a></sup>.</li>
<li>Specifically, \(\rootfactor(g, (e_i)_{i\in[n]}) = (w_i)_{i\in[n]} = (a^{1/e_i})_{i\in[n]} = \left((g^{\prod_{j\in[n]} e_j})^{1/e_i}\right)_{i\in[n]}\)</li>
</ul>
</li>
<li><a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html">Catalano-Fiore Vector Commitments</a>
<ul>
<li>Let $H$ be a collision-resistant hash function that maps a vector position $i$ to an $\ell+1$ bit prime $e_i$ such that $2^\ell < e_i < 2^{\ell+1}$</li>
<li>The digest of a vector $\vect{v} = [v_1, \dots, v_n]$ is $d(\vect{v}) = (S, \Lambda)$ where:
<ul>
<li>$S = g^{\prod_{i\in[n]} e_i}$ (i.e., an RSA accumulator over all vector indices $i$)</li>
<li>$\Lambda = \prod_{i\in [n]} (S^{1/e_i})^{v_i}$</li>
<li>Note that $\Lambda$ is a multi-exponentiation, where:
<ul>
<li>The bases are RSA witness $S^{1/e_i}$ for $i$,</li>
<li>The exponents are the elements $v_i$!</li>
</ul>
</li>
</ul>
</li>
<li>A proof $\pi_I = (S_I, \Lambda_I)$ for an $I$-subvector $(v_i)_{i\in I}$ is just the digest of $\vect{v}$ without the positions $i\in I$ in it.
<ul>
<li>$S_I = S^{1/\prod_{i\in I} e_i}$ (i.e., an RSA accumulator over all indices except the ones in $I$)</li>
<li>$\Lambda_I = \prod_{i\in[n]\setminus I} (S_I^{1/e_i})^{v_i}$</li>
<li>Again, note that $\Lambda_I$ is a multi-exponentiation, where:
<ul>
<li>The bases are RSA witnesses $S_I^{1/e_i}$ for each $i\in[n]\setminus I$ (but w.r.t. $S_I$)</li>
<li>The exponents are elements $v_i$ for all $i\in[n]\setminus I$</li>
</ul>
</li>
</ul>
</li>
<li>Digests and proofs are <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html#updating-digest">updatable</a></li>
<li>Proofs are <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html#disaggregating-proofs">incrementally <em>(dis)aggregatable</em></a></li>
</ul>
</li>
</ul>
<h2 id="authenticated-dictionary-ad-schemes">Authenticated dictionary (AD) schemes</h2>
<p>First, forget about <em>authenticated dictionaries</em> and let’s talk about good old plain <em>dictionaries</em>!
Dictionaries are a set of <strong>key-value pairs</strong> such that each <strong>key</strong> is mapped to one <strong>value</strong>.
(We stick to one value per key here, but one could define dictionaries to have multiple values per key too.)
The keys are elements of a <strong>key space</strong> which, for our purposes, is the set of strings of length $2\lambda$ bits.</p>
<p class="info"><strong>Example:</strong> Your phone’s contacts list is a dictionary: it maps each contact’s phone number (i.e., the key) to that contact’s name (i.e., the value).
Similarly, your French-to-English dictionary maps each French word (i.e., the key) to its English counterpart (i.e., the value).</p>
<p>Second, what does it mean to <em>authenticate</em> a dictionary?
The idea is to outsource storage of the dictionary to a <strong>prover</strong> while allowing one or more <strong>verifiers</strong> to correctly <strong>look up</strong> the values of keys in the dictionary.
For this to work, the verifiers must be able to somehow verify the values of keys claimed by the prover.
It should be clear that if the verifiers store nothing, there is nothing they can verify these claims against.
Thus, verifiers must store <em>something</em>.
Since the goal is to outsource storage of the data structure, verifiers will only store a succinct representation of the dictionary called a <strong>digest</strong>.
Importantly, while the dictionary might be very large (e.g., the contacts list of a social butterfly), the digest will actually be constant-sized (e.g., 32 bytes).</p>
<p>Third, how do verifiers look up in authenticated dictionaries?
Verifiers simply ask the prover for a key’s value!
Then, the prover replies with the value together with a <strong>lookup proof</strong> that the verifier checks against the digest!</p>
<p class="info"><strong>Example:</strong> Some of you might be familiar with Merkle prefix trees.
Consider a “sparse” prefix tree that maps each key to a unique leaf.
This is best explained by Laurie and Kasper<sup id="fnref:LK15" role="doc-noteref"><a href="#fn:LK15" class="footnote">4</a></sup> but, simply put, each key is hashed to a unique path in the tree whose leaf stores that key’s value.
The <em>digest</em> is the Merkle root hash of this Merkle prefix tree.
A <em>lookup proof</em> is the Merkle sibling path to the key’s value in the prefix tree.</p>
<h2 id="our-updatable-ad-for-stateless-validation">Our updatable AD for stateless validation</h2>
<p>We start with a <strong>simple observation</strong>: the CF VC scheme can be repurposed into an authenticated dictionary scheme by treating the vector indices as the dictionary’s keys<sup id="fnref:obs1" role="doc-noteref"><a href="#fn:obs1" class="footnote">5</a></sup>.
Recall that CF VCs use a collision-resistant hash function $H$ that maps a vector position $i$ to an $(\ell+1)$-bit prime $e_i$ such that $2^\ell < e_i < 2^{\ell+1}$.</p>
<p>We let $e_k = H(k)$ for each key $k$ in the dictionary.
Then, the dictionary’s digest is:
\begin{align}
S &= g^{\prod_{k\in D} e_k}\\<br />
c &= \prod_{(k,v) \in D} (S^{1/e_k})^v
\end{align}
Note that this is just a CF commitment to a “very sparse” vector, with indices in the key space of the dictionary
(The key space is of size $2^{2\lambda}$ since it contains all strings of length $2\lambda$ bits.)
In other words, the dictionary’s key is the vector’s index while the key’s value is the vector element at that index.
Because of this, all the properties of CF VCs carry over to our authenticated dictionary: constant-sized public parameters, incremental proof (dis)aggregation, proof updates and proof precomputation.</p>
<p>Nonetheless, we further enhance this AD by making it more updatable and more (dis)aggregatable.
We call the resulting AD an <strong>updatable authenticated dictionary (UAD)</strong>.</p>
<p class="info">Note that ADs cannot be obtained in this fashion from any VC scheme.
For example, <a href="https://alinush.github.io/2020/05/06/aggregatable-subvector-commitments-for-stateless-cryptocurrencies.html">KZG-based VCs</a> do not support a sparse set of vector indices (but nonetheless other techniques<sup id="fnref:Feis20Multi" role="doc-noteref"><a href="#fn:Feis20Multi" class="footnote">6</a></sup> can be used there).
However, some schemes like Catalano-Fiore<sup id="fnref:CF13e" role="doc-noteref"><a href="#fn:CF13e" class="footnote">7</a></sup> and Boneh et al’s VC<sup id="fnref:BBF18:1" role="doc-noteref"><a href="#fn:BBF18" class="footnote">3</a></sup> do support sparse indices.
Indeed, Boneh et al.<sup id="fnref:BBF18:2" role="doc-noteref"><a href="#fn:BBF18" class="footnote">3</a></sup> also build an AD on top of their VC scheme, but it is not as (dis)aggregatable as ours.</p>
<h3 id="updating-the-digest-after-removals">Updating the digest after removals</h3>
<p>One new feature we add is updating the digest after a key and its value are <em>removed</em> from the dictionary.
This is very easy to do thanks to the versatility of CF VCs.
First, recall that the proof for $(k,v)$ is just the digest of the dictionary $D$ but without $(k,v)$ in it.
Thus, if we remove $(k,v)$ from $D$, the new digest is just the proof for $(k,v)$!
If we do multiple removals, we can simply <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html#aggregating-proofs">aggregate</a> the proofs of all removed keys, which is just the digest of $D$ without those keys in it.
Thus, the new digest after multiple removals is simply this aggregated proof!</p>
<h3 id="updating-proofs-after-removals">Updating proofs after removals</h3>
<p>We also have to add support for updating proofs after a key (and its value) is <em>removed</em> from the dictionary.
Let’s say we want to update an aggregated proof $\pi_K$ for a set of keys $K$ after removing a single key $\hat{k}$ with proof $\pi_{\hat{k}}$.
Recall that $\pi_K$ is the digest of $D\setminus K$.
Since the updated dictionary will be \(D\setminus \{\hat{k}\}\), the updated proof $\pi_K’$ must be the digest of \((D\setminus \{\hat{k}\}) \setminus K\), which is just \(D\setminus (\{\hat{k}\}\cup K)\).</p>
<p>So we must find a way to go from the digest of $D\setminus K$ and of \(D\setminus\{\hat{k}\}\) to the digest of \(D\setminus (\{\hat{k}\}\cup K)\).
Well, the digest of \(D\setminus (\{\hat{k}\}\cup K)\) is nothing but the aggregated proof for $K$ and $\hat{k}$.
Thus, the updated proof for $K$ is simply the aggregation of the old proof for $K$ with the proof for the removed $\hat{k}$.
Naturally, if multiple keys are being removed, then we just aggregate $\pi_K$ with the proofs for each removed key.</p>
<p class="warning">One thing we’ve glanced over was that if \(K = \{\hat{k}\}\), then this proof update doesn’t really work, since we’d be updating the proof for $\hat{k}$ after removing $\hat{k}$ itself.
This doesn’t make sense unless we updated $\hat{k}$’s lookup proof into a non-membership proof, which we have not defined yet, but will do so in a future post.
<br />
<br />
We’ve also glanced over having $\hat{k}\in K$.
But this is not problematic since, in this case, we have \(D\setminus K = D\setminus (\{\hat{k}\}\cup K)\), so the updated proof $\pi_K’ =\pi_K$.</p>
<h3 id="cross-incremental-proof-aggregation">Cross-incremental proof aggregation</h3>
<p>Our paper’s main contribution is <em>cross-incremental proof aggregation</em> for our AD, a technique for <strong>incrementally</strong> aggregating lookup proofs <em>across different dictionaries</em>.
Recall that we can already (incrementally) aggregate two proofs, one for a set of keys $K_1$ and another for $K_2$, into a single proof for the set of keys $K_1\cup K_2$.
For this to work though, these two proofs must be w.r.t. the same dictionary digest $d$.
However, in some applications, we’ll be dealing with proofs $\pi_i$, each for a set of keys $K_i$ but w.r.t. their own digest $d_i$.
This raises the question of whether such proofs can also be <em>cross-aggregated</em>?
Gorbunov et al.<sup id="fnref:GRWZ20e" role="doc-noteref"><a href="#fn:GRWZ20e" class="footnote">8</a></sup> answer this question positively for vector commitments and our work extends this to authenticated dictionaries.</p>
<p class="info"><strong>Example:</strong> In stateless validation for smart contracts<sup id="fnref:GRWZ20e:1" role="doc-noteref"><a href="#fn:GRWZ20e" class="footnote">8</a></sup>, the $i$th’s smart contract’s memory is represented as a dictionary with digest $d_i$.
When this $i$th contract is invoked, the transaction will need to include the subset of memory locations $K_i$ that were accessed by the execution together with their proof $\pi_i$.
When multiple transactions are processed, each proof $\pi_i$ will be w.r.t. a different $d_i$.
Importantly, instead of including each $\pi_i$ in the mined block, we would ideally like to <em>cross-aggregate</em> all $\pi_i$’s into a single proof $\pi$.</p>
<h4 id="proof-of-knowledge-of-co-prime-roots">Proof-of-knowledge of co-prime roots</h4>
<p>The key ingredient behind our incremental cross-aggregation is the <strong>proof-of-knowledge of co-prime roots (PoKCR)</strong> protocol by Boneh et al.<sup id="fnref:BBF18:3" role="doc-noteref"><a href="#fn:BBF18" class="footnote">3</a></sup>
Recall that PoKCR can be used to convince a verifier who has $\alpha_i$’s and $x_i$’s, that the prover <em>knows</em> $w_i$’s such that:</p>
\[\alpha_i = w_i^{x_i},\ \text{for each}\ i\in[n]\]
<p>Importantly, this protocol requires that the $x_i$’s are pairwise co-prime:</p>
\[\gcd(x_i, x_j) = 1,\forall i,j\in[n], i\ne j\]
<p>To prove knowledge of the $w_i$’s, the prover simply gives the verifier:</p>
\[W=\prod_{i\in [n]} w_i\]
<p>To verify knowledge of $w_i$’s, the verifier (who has $\alpha_i$’s and $x_i$’s) computes \(x^* = \prod_{i\in[n]} x_i\) and checks if:</p>
\[W^{x^*} \stackrel{?}{=} \prod_{i\in [n]} \alpha_i^{x^*/x_i}\]
<p>The trick for the verifier is to do this computation efficiently, since the right-hand side (RHS) involves $n$ exponentiations, each of size $O(\ell n)$ bits.
If done naively, this would take $O(\ell n^2)\ \Gho$ operations.
Fortunately, Boneh et al.<sup id="fnref:BBF18:4" role="doc-noteref"><a href="#fn:BBF18" class="footnote">3</a></sup> give an $O(\ell n\log{n})$ time algorithm to compute this RHS denoted by:</p>
\[\multirootexp((\alpha_i, x_i)_{i\in [n]}) = \prod_{i\in [n]} \alpha_i^{x^*/x_i}\]
<p>We refer you to Figure 1 in our paper<sup id="fnref:TXN20e:1" role="doc-noteref"><a href="#fn:TXN20e" class="footnote">1</a></sup> for the $\multirootexp$ algorithm, which simply leverages the recursive nature of the problem.
In fact, the algorithm recurses in a manner very similar to <a href="/2020/11/24/RSA-Accumulators.html#precomputing-all-membership-witnesses-fast">$\rootfactor$</a>.</p>
<p>Importantly, Boneh et al. give an <em>extractor</em> that the PoKCR verifier can use to actually recover the $w_i$’s from the $x_i$’s, $\alpha_i$’s and $W$.
This is what makes the protocol a proof of <em>knowledge</em>.
One of our contributions is speeding up the extraction of <em>all</em> $w_i$’s from $O(\ell n^2\log{n})\ \Gho$ operations down to $O(\ell n\log^2{n})$<sup id="fnref:obs2" role="doc-noteref"><a href="#fn:obs2" class="footnote">9</a></sup>.
For this, we refer you to our full paper<sup id="fnref:TXN20e:2" role="doc-noteref"><a href="#fn:TXN20e" class="footnote">1</a></sup>.</p>
<h4 id="using-pokcr-for-incrementally-cross-aggregating-lookup-proofs">Using PoKCR for incrementally cross-aggregating lookup proofs</h4>
<p>Suppose we have a lookup proof $\pi_i$ for a set of keys $K_i$ in a dictionary $D_i$ with digest $d_i = (A_i, c_i)$, where $A_i$ is the RSA accumulator over all keys in the dictionary and $c_i$ is the multi-exponentiation of RSA witnesses (i.e., the part of the proof previously denoted using $\Lambda$).
Note we are changing notation slightly for ease of presentation.</p>
<p>The main observation is that we can aggregate several proofs $\pi_i = (W_i, \Lambda_i)$ w.r.t. different digests $d_i$ via PoKCR because $W_i$ and $\Lambda_i$ are actually prime roots of certain group elements.
To see this, recall from the <a href="#preliminaries">preliminaries</a> that:</p>
<p>\begin{align}
W_i &= A_i^{1/e_{K_i}}\\<br />
\Lambda_i &= \left(\prod_{(k,v)\in D_i\setminus K_i} (A_i^{1/e_k})^{v}\right)^{1/e_{K_i}}
\end{align}</p>
<p>Clearly, $W_i$ is an $e_{K_i}$-th root of $A_i$, which the verifier has.
But what about $\Lambda_i$?
Let $v_k$ be the value of each $k\in K_i$ and rewrite $\Lambda_i$ as:
\begin{align}
\Lambda_i &= \left(\prod_{(k,v)\in D_i\setminus K_i} (A_i^{1/e_k})^{v}\right)^{1/e_{K_i}}\\<br />
&= \left(\frac{\prod_{(k,v)\in D_i} (A_i^{1/e_k})^{v}}{\prod_{k\in K_i} (A_i^{1/e_k})^{v_k}}\right)^{1/e_{K_i}}\\<br />
&= \left(c_i / \prod_{k\in K_i} (A_i^{1/e_k})^{v_k}\right)^{1/e_{K_i}}\\<br />
\end{align}
Thus, if we let \(\alpha_i = c_i / \prod_{k\in K_i} (A_i^{1/e_k})^{v_k}\), then $\Lambda_i$ is an $e_{K_i}$-th root of $\alpha_i$.
Note that the verifier can compute $\alpha_i$ from $c_i, W_i$ and $K_i$ (as we describe later).</p>
<p>To summarize, we have $m$ proofs $\pi_i = (W_i, \Lambda_i)$ each w.r.t. its own $d_i = (A_i, c_i)$ such that, for all $i\in [m]$:
\begin{align}
W_i^{e_{K_i}} &= A_i\\<br />
\Lambda_i^{e_{K_i}} &= \alpha_i
\end{align}</p>
<p>We are almost ready to aggregate with PoKCR, but we cannot yet.
This is because the $e_{K_i}$’s must be pairwise co-prime for PoKCR to work!
However, this is not necessarily the case, since we could have a key $k$ that is both in $K_i$ and in $K_j$ which means $e_{K_i}$ and $e_{K_j}$ will have a common factor $e_k = H(k)$.</p>
<p>Fortunately, we can quickly work around this by using a different hash function $H_i$ for each dictionary $D_i$.
This way, the prime representatives for $k\in K_i$ are computed as $e_k = H_i(k)$, while the prime representatives for $k\in K_j$ are computed as $e_k = H_j(k)$.
As long as one cannot find any pair $(k,k’)$ with $H_i(k) = H_j(k’)$, all the $e_{K_i}$’s will be pairwise co-prime.
This means we can aggregate all $m$ proofs as:
\begin{align}
W &= \prod_{i\in[m]} W_i\\<br />
\Lambda &= \prod_{i\in [m]} \Lambda_i
\end{align}</p>
<p>Importantly, we can do this aggregation <em>incrementally</em>: whenever a new proof arrives, we simply multiply it in the previously cross-aggregated proof.</p>
<h4 id="verifying-cross-aggregated-lookup-proofs">Verifying cross-aggregated lookup proofs</h4>
<p>Suppose a verifier gets a cross-aggregated proof $\pi = (W,\Lambda)$ for a bunch of $K_i$’s w.r.t. their own $d_i = (A_i, c_i),\forall i\in[m]$.
How can he verify $\pi$?
First, the verifier checks the PoKCR that, for each $i\in[m]$, there exists $W_i$ such that $A_i = W_i^{e_{K_i}}$:</p>
\[W^{e^*} \stackrel{?}{=} \multirootexp((A_i, e_{K_i})_{i\in [m]}) = \prod_{i\in [m]} A_i^{e^*/e_{K_i}}\]
<p>Here, $e^*=\prod_{i\in[m]} e_{K_i}$ and each $e_{K_i} = \prod_{k\in K_i} H_i(k)$.
Importantly, the verifier can recover the $W_i$’s using the PoKCR extractor (see Section 3.1 in our full paper<sup id="fnref:TXN20e:3" role="doc-noteref"><a href="#fn:TXN20e" class="footnote">1</a></sup>).</p>
<p>Second, the verifier checks the PoKCR for each $\alpha_i = \Lambda_i^{e_{K_i}}$.
For this, the verifier must first compute each $\alpha_i = c_i / \prod_{k\in K_i} (A_i^{1/e_k})^{v_k}$, where $v_k$ is the value of each $k \in K_i$ and $e_k = H_i(k)$.
The difficult part is computing all $A_i^{1/e_k}$’s, but this can be done via $\rootfactor(W_i, (e_k)_{k\in K_i})$.
Once the verifier has the $\alpha_i$’s, he can check:</p>
\[\Lambda^{e^*} \stackrel{?}{=} \multirootexp((\alpha_i, e_{K_i})_{i\in [m]}) = \prod_{i\in [m]} \alpha_i^{e^*/e_{K_i}}\]
<p>If both PoKCR checks pass, then the verifier is assured the proof verifies.
Not only that, but the verifier can also disaggregate the cross-aggregated proof as we explain next.</p>
<h4 id="disaggregating-cross-aggregated-proofs">Disaggregating cross-aggregated proofs</h4>
<p>Since the cross-aggregated proof $\pi = (W,\Lambda)$ is a PoKCR proof, this mean the PoKCR extractor can be used to recover the original proofs $(\pi_i)_{i\in[m]}$ that $\pi$ was aggregated from.
How?</p>
<p>Well, we already showed how the verifier must extract the $W_i$’s in the original proofs, which he needs for reconstructing the $\alpha_i$’s to verify the $\Lambda$ part of the cross-aggregated proof.
In a similar fashion, the verifier can also extract all the $\Lambda_i$’s aggregated in $\Lambda$.
This way, the verifier can recover the original proofs.
Note that this implies cross-aggregated proofs are <em>updatable</em> by:</p>
<ol>
<li>Cross-disaggregating them into the original lookup proofs,</li>
<li>Updating these lookup proofs,</li>
<li>And cross-reaggregating them back.</li>
</ol>
<h2 id="conclusion">Conclusion</h2>
<p>To conclude, we show that generalizing CF to a larger key-space results in a versatile <em>authenticated dictionary (AD)</em> scheme that supports updating proofs and digests and supports aggregating proofs across different dictionaries in an incremental fashion.
In a future post, we strengthen the security of this construction, which makes it applicable to more adversarial applications such as transparency logging.
As always, see our full paper for details<sup id="fnref:TXN20e:4" role="doc-noteref"><a href="#fn:TXN20e" class="footnote">1</a></sup>.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:TXN20e" role="doc-endnote">
<p><strong>Authenticated Dictionaries with Cross-Incremental Proof (Dis)aggregation</strong>, by Alin Tomescu and Yu Xia and Zachary Newman, <em>in Cryptology ePrint Archive, Report 2020/1239</em>, 2020, <a href="https://eprint.iacr.org/2020/1239">[URL]</a> <a href="#fnref:TXN20e" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:TXN20e:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a> <a href="#fnref:TXN20e:2" class="reversefootnote" role="doc-backlink">↩<sup>3</sup></a> <a href="#fnref:TXN20e:3" class="reversefootnote" role="doc-backlink">↩<sup>4</sup></a> <a href="#fnref:TXN20e:4" class="reversefootnote" role="doc-backlink">↩<sup>5</sup></a></p>
</li>
<li id="fn:SSY01" role="doc-endnote">
<p><strong>Blind, Auditable Membership Proofs</strong>, by Sander, Tomas and Ta-Shma, Amnon and Yung, Moti, <em>in Financial Cryptography</em>, 2001 <a href="#fnref:SSY01" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:BBF18" role="doc-endnote">
<p><strong>Batching Techniques for Accumulators with Applications to IOPs and Stateless Blockchains</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/1188</em>, 2018, <a href="https://eprint.iacr.org/2018/1188">[URL]</a> <a href="#fnref:BBF18" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:BBF18:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a> <a href="#fnref:BBF18:2" class="reversefootnote" role="doc-backlink">↩<sup>3</sup></a> <a href="#fnref:BBF18:3" class="reversefootnote" role="doc-backlink">↩<sup>4</sup></a> <a href="#fnref:BBF18:4" class="reversefootnote" role="doc-backlink">↩<sup>5</sup></a></p>
</li>
<li id="fn:LK15" role="doc-endnote">
<p><strong>Revocation Transparency</strong>, by Ben Laurie and Emilia Kasper, 2015, <a href="https://www.links.org/files/RevocationTransparency.pdf">[URL]</a> <a href="#fnref:LK15" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:obs1" role="doc-endnote">
<p>We were not the first to make this observation; see the work by Agrawal and Raghuraman<sup id="fnref:AR20" role="doc-noteref"><a href="#fn:AR20" class="footnote">10</a></sup> <a href="#fnref:obs1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Feis20Multi" role="doc-endnote">
<p><strong>Multi-layer hashmaps for state storage</strong>, by Dankrad Feist, 2020, <a href="https://ethresear.ch/t/multi-layer-hashmaps-for-state-storage/7211/print">[URL]</a> <a href="#fnref:Feis20Multi" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:CF13e" role="doc-endnote">
<p><strong>Vector Commitments and their Applications</strong>, by Dario Catalano and Dario Fiore, <em>in Cryptology ePrint Archive, Report 2011/495</em>, 2011, <a href="https://eprint.iacr.org/2011/495">[URL]</a> <a href="#fnref:CF13e" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:GRWZ20e" role="doc-endnote">
<p><strong>Pointproofs: Aggregating Proofs for Multiple Vector Commitments</strong>, by Sergey Gorbunov and Leonid Reyzin and Hoeteck Wee and Zhenfei Zhang, 2020, <a href="https://eprint.iacr.org/2020/419">[URL]</a> <a href="#fnref:GRWZ20e" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:GRWZ20e:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:obs2" role="doc-endnote">
<p>See Section 3.1 in our full paper<sup id="fnref:TXN20e:5" role="doc-noteref"><a href="#fn:TXN20e" class="footnote">1</a></sup> <a href="#fnref:obs2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:AR20" role="doc-endnote">
<p><strong>KVaC: Key-Value Commitments for Blockchains and Beyond</strong>, by Shashank Agrawal and Srinivasan Raghuraman, <em>in Cryptology ePrint Archive, Report 2020/1161</em>, 2020, <a href="https://eprint.iacr.org/2020/1161">[URL]</a> <a href="#fnref:AR20" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Alin Tomescutl;dr: We build an authenticated dictionary (AD) from Catalano Fiore vector commitments that has constant-sized, aggregatable proofs and supports a stronger notion of cross-incremental proof disaggregation. Our AD could be used for stateless validation in cryptocurrencies with smart contract execution. In a future post, we will extend this AD with stronger security, non-membership proofs and append-only proofs, which makes it applicable to transparency logging. This is joint work with my brilliant (ex-)labmates from MIT, Alex (Yu) Xia and Zack Newman. Authenticated dictionaries (ADs) are an important cryptographic primitive which lies at the core of cryptocurrencies such as Ethereum and of transparency logs such as Certificate Transparency (CT). Typically, ADs are constructed by Merkleizing a lexicographically-ordered data structure such as a binary search tree, a prefix tree or a skip list. However, our work takes a different, more algebraic direction, building upon the Catalano-Fiore (CF) vector commitment (VC) scheme. This has the advantage of giving us constant-sized proofs which are updatable and aggregatable, with a novel notion of cross-incrementality. Importantly, this combination of feautres is not supported by Merkle trees or any other previous VC scheme.Catalano-Fiore Vector Commitments2020-11-24T00:00:00+00:002020-11-24T00:00:00+00:00https://alinush.github.io//2020/11/24/Catalano-Fiore-Vector-Commitments<!-- TODO: Write an intro paragraph here -->
<p>A <em>vector commitment (VC)</em> scheme allows a <strong>prover</strong> with access to a vector $\mathbf{v} = [ v_1, \dots, v_n ]$ to convince any <strong>verifier</strong> that position $i$ in $\mathbf{v}$ stores $v_i$ for any index $i\in[n]$.
Importantly, verifers only store a succinct <strong>digest</strong> of the vector (e.g., a 32-byte hash) rather than the full vector $\mathbf{v}$.</p>
<!--more-->
<p hidden="">$$
\def\Adv{\mathcal{A}}
\def\Badv{\mathcal{B}}
\def\GenGho{\mathsf{GenGroup}_?}
\def\Ghosz{|\Gho|}
\def\Ghoid{1_{\Gho}}
\def\multirootexp{\mathsf{MultiRootExp}}
\def\primes{\mathsf{Primes}}
\def\QRn{\mathsf{QR}_N}
\def\rootfactor{\mathsf{RootFactor}}
\def\shamirtrick{\mathsf{ShamirTrick}}
\def\vect#1{\mathbf{#1}}
$$</p>
<p>Below, we review <em>Catalano and Fiore’s</em> elegant VC scheme<sup id="fnref:CF13e" role="doc-noteref"><a href="#fn:CF13e" class="footnote">1</a></sup> built from hidden-order groups, extended with the enhancements proposed by Lai and Malavolta<sup id="fnref:LM18" role="doc-noteref"><a href="#fn:LM18" class="footnote">2</a></sup> and by Campanelli et al.<sup id="fnref:CFGplus20e" role="doc-noteref"><a href="#fn:CFGplus20e" class="footnote">3</a></sup>
We also make one new, small observation: <a href="#disaggregating-proofs">proof disaggregation</a> in this VC is faster than originally thought, which helps precompute all proofs slightly faster.</p>
<h2 id="preliminaries">Preliminaries</h2>
<p>Let $[n] = \{1,2,\dots, n\}$ and $[a,b] = \{a, a+1,\dots, b-1, b\}$.</p>
<p>We assume the reader is familiar with <a href="/2020/11/24/RSA-accumulators.html">RSA accumulators</a>, which Catalano-Fiore (CF) VCs rely heavily upon.
Also, we assume familiarity with the <a href="/2020/11/24/RSA-accumulators.html#precomputing-all-membership-witnesses-fast">$\rootfactor$ algorithm</a> for computing <strong>all</strong> $e_j$th roots of $g^{\prod_{i\in[n]} e_i}, j\in [n]$.</p>
<h3 id="shamirs-trick">Shamir’s trick</h3>
<p>Given $z^{1/a}$ and $z^{1/b}$ where $z\in\Gho$ and $a,b\in\Z$ with $\gcd(a,b) = 1$, one can compute $z^{1/ab} = (z^{1/a})^y (z^{1/b})^x$ where $x,y$ are the Bezout coefficients for $a,b$ such that $ax+by=1$.</p>
<p>This is because:</p>
\[(z^{1/a})^y (z^{1/b})^x = z^\frac{by}{ab} z^\frac{ax}{ab} = z^\frac{ax+by}{ab} = z^\frac{1}{ab}\]
<p>We often use $z^\frac{1}{ab}\leftarrow \shamirtrick(z^{1/a}, z^{1/b}, a, b)$ to denote performing a Shamir’s trick..</p>
<p>Computing $z^{1/(ab)}$ takes \(O(\max(\vert x\vert,\vert y\vert))\ \Gho\) operations since the Bezout coefficients $(a,b)$ are as big as $(x,y)$.
(Some extra time is needed to compute the Bezout coefficients, but we ignore it here.)</p>
<h2 id="public-parameters">Public parameters</h2>
<p>To set up the VC scheme, a hidden-order group $\Gho$ with generator $g$ must be picked such that nobody knows the order of the group.
Importantly, the RSA problem needs to be hard in this group.
<!-- TODO: reference assumptions -->
Typically, $\Gho = \ZNs$ where $N=pq$ is picked via a secure <em>multi-party computation (MPC) ceremony</em> such that nobody knows the factorization of $N$ (and thus nobody knows the order of $\Gho$; i.e., $\vert\Gho\vert = \phi(N)= (p-1)(q-1)$).</p>
<p>Then, a collision-resistant hash function $H : [n] \rightarrow \primes^{\ell+1}$ must be fixe.
$H$ will map each vector index $i\in [n]$ to a prime $e_i = H(i)$ such that $2^\ell < e_i < 2^{\ell+1}$ where $\ell$ is the maximum size in bits of a vector element $v_i$.
We often use $e_K = \prod_{k\in K} e_k$ to denote a product of a subset of such primes, where $K\subseteq[n]$.</p>
<h2 id="digest-or-commitments">Digest (or commitments)</h2>
<p>Instead of commitments, we’ll stick to the <em>digest</em> terminology.</p>
<p>The digest of a vector $\vect{v} = [ v_1, \dots, v_n ]$ consists of two parts.
First, an <a href="/2020/11/24/RSA-accumulators.html">RSA accumulator</a> over all positions in the vector:
\begin{align}
S &= g^{\prod_{i\in[n]} e_i}
\end{align}</p>
<p>Second, a multi-exponentiation of each <a href="/2020/11/24/RSA-accumulators.html#membership-witnesses">RSA membership witness</a> (w.r.t. $S$) for position $i$ to the value $v_i$:
\begin{align}
\Lambda &= \prod_{i\in [n]} (S^{1/e_i})^{v_i}
\end{align}</p>
<p>We often use $d(\vect{v}) = (S, \Lambda)$ to denote the digest of a vector.</p>
<p class="info">Computing the digest requires computing all $S^{1/e_i}, \forall i\in[n]$.
As described <a href="/2020/11/24/RSA-accumulators.html#precomputing-all-membership-witnesses-fast">before</a>, this can be done in $O(n\log{n})$ group exponentiations via \((S^{1/e_i})_{i\in[n]} \leftarrow \rootfactor(g, (e_i)_{i\in[n]})\).
Since each exponentiation is by an $(\ell+1)$-bit prime $e_i$, this takes $O(\ell n\log{n})\ \Gho$ operations.
Then, $\Lambda$ can be computed with an additional $O(\ell n)\ \Gho$ operations.</p>
<p>The beauty of CF VCs is that, once you understand what a digest looks like, everything follows very naturally.</p>
<h2 id="proofs-are-just-digests">Proofs are just digests</h2>
<p>At minimum, in any VC scheme, a verifier can be convinced of the value $v_i$ of any position $i$ in the vector using a <strong>proof</strong> $\pi_i$.
To convince the verifier of multiple values $(\vect{v}_i)_{i\in I}$, this would require individual proofs for each position $i\in I$.
However, in VCs like CF, a single $I$-subvector proof $\pi_I$ can be constructed for all $(v_i)_{i\in I}$.
Importantly, the size of $\pi_I$ is always constant, independent of $\vert I\vert$.</p>
<p>In CF, an $I$-subvector proof $\pi_I$ is just the digest of the vector $\vect{v}$ but <em>“without”</em> positions $I$ in it.
We abuse notation and denote this by \(\vect{v} \setminus I = (v_i)_{i\in[n] \setminus I}\).
Thus, $\pi_I = d(\vect{v}\setminus I)$.</p>
<p>Let $e_I = \prod_{i\in I} e_i$.
Then, the proof $\pi_I$ is just:
\begin{align}
S_I &= S^{1/e_I} = g^{\prod_{i\in[n]\setminus I} e_i}\\<br />
\Lambda_I &= \prod_{i\in [n]\setminus I} (S_I^{1/e_i})^{v_i}
\end{align}</p>
<p>Observe that the accumulator $S_I$ is over all positions $i$ except the ones in $I$.
Also, observe that the membership witnesses in $\Lambda_I$ are w.r.t. to $S_I$ (not $S$) and that $\Lambda_I$ does not contain witnesses for $i\in I$.
This is what it means to commit to a vector $\vect{v}$ “without” positions $I$ in it.</p>
<p class="info">Computing $\pi_I$ takes $O(\ell(n-\vert I \vert)\log(n-\vert I\vert))\ \Gho$ operations, dominated by the cost to compute \((S_I^{1/e_i})_{i\in I}\) via \(\rootfactor(g, (e_i)_{i\in[n]\setminus I})\).</p>
<p>To verify the proof, one simply “adds back” \(\vect{v}_I\) to \(d(\vect{v}\setminus I) = \pi_I\) and checks it obtains the digest $d(\vect{v}) = (S, \Lambda)$ of the vector:
\begin{align}
S &\stackrel{?}{=} S_I^{e_I}\\<br />
\Lambda &\stackrel{?}{=} \Lambda_I^{e_I} \prod_{i\in I} (S^{1/e_i})^{v_i}
\end{align}</p>
<p class="warning">One problem is that all \((S^{1/e_i})_{i\in I}\) must be computed via \(\rootfactor(S_I, (e_i)_{i\in I})\), which would take $O(\ell \vert I\vert \log\vert I\vert)\ \Gho$ operations.
We explain <a href="#updating-digest">next</a> how this can be done in $O(\ell \vert I \vert)\ \Gho$ operations by adding each $v_i, i\in I$ back into $\pi_I$ <em>sequentially</em>!</p>
<h2 id="updating-digest">Updating digest</h2>
<p>It is possible to update the digest $d(\vect{v})=(S,\Lambda)$ after an element $v_i$ changes to $v_i + \delta_i$.
For this, the RSA membership witness $S^{1/e_i}$ is needed as helper information, which we refer to as an <strong>update key</strong>.</p>
<p>Since $S$ does not contain any information about $v_i$ (just about $i$), $S$ stays the same.
However, $\Lambda$ must change into $\Lambda’$ as follows:
\begin{align}
\Lambda’ = \Lambda \cdot (S^{1/e_i})^{\delta_i}
\end{align}</p>
<p class="info">This technique can be generalized to work with many positions $i\in I$ changing by $\delta_i$.
This either requires all the $S^{1/e_i}$ update keys or an <strong>aggregated update key</strong> \(S_I = S^{1/e_I} = S^{\prod_{i\in I} 1/e_i}\) as auxiliary information.
In this last case, all the \((S^{1/e_i})_{i\in I} \leftarrow \rootfactor(S_I, (e_i)_{i\in I})\) can be computed in $O(\ell \vert I \vert \log \vert I \vert)\ \Gho$ operations from the aggregated update key.
Then the update can be performed as \(\Lambda' = \Lambda \cdot \prod_{i\in I} (S^{1/e_i})^{\delta_i}\)</p>
<h3 id="adding-new-positions-to-the-vector">Adding new positions to the vector</h3>
<p>It is also possible to “extend” the vector $\vect{v}$ of size $n$ to size $n+1$ by adding an extra element $v_{n+1}$.
For this, the digest can be updated as:
\begin{align}
S’ &= S^{e_{n+1}}\\<br />
\Lambda’ &= \Lambda^{e_{n+1}} S^{v_{n+1}} = \dots = \prod_{i\in [n+1]} (S’^{1/e_i})^{v_i}
\end{align}</p>
<p class="info">This can be generalized to adding $\Delta$ new positions, and would take $O(\ell \Delta)\ \Gho$ operations by applying each extension sequentially.
In fact, this how one would verify a subvector proof fast: by adding back the elements being verified to the proof (which can be viewed as a digest) and checking the actual digest is obtained!</p>
<h2 id="updating-proofs">Updating proofs</h2>
<p>Since proofs are just digests they are, in principle, updatable in the same fashion, with a few extra details.</p>
<p>Suppose we have a proof $\pi_I$ for a subvector \(\vect{v}_I\):
\begin{align}
S_I &= S^{1/e_I} = S^{1/\prod_{i\in I} e_i} = g^{\prod_{i\in[n]\setminus I} e_i}\\<br />
\Lambda_I &= \prod_{i\in [n]\setminus I} (S_I^{1/e_i})^{v_i}
\end{align}</p>
<h3 id="case-1-iin-i-changed">Case 1: $i\in I$ changed</h3>
<p>First, suppose a position $i\in I$ changes from \(v_i\) to \(v_i + \delta_i\).
Then, the proof $\pi_I$ remains the same!
This is because the proof contains no information about \(v_i\), since the proof is the digest of $\vect{v}$ without any positions $i\in I$.</p>
<h3 id="case-2-jnotin-i-changed">Case 2: $j\notin I$ changed</h3>
<p>Second, suppose a position $j\notin I$ changes from \(v_j\) to \(v_j+\delta_j\).
In this case, we only need to change $\Lambda_I$ into a $\Lambda_I’$ such that:
\begin{align}
\Lambda_I’ = \Lambda_I \cdot (S_I^{1/e_j})^{\delta_j}
\end{align}</p>
<p>The difficult part is computing $S_I^{1/e_j}$.
We could do it from scratch but that would take $O(\ell (n - \vert I \vert))\ \Gho$ operations.
Instead, as with the <a href="#updating-digest">digest update</a>, we assume we are given the $S^{1/e_j}$ update key associated with the changed position $j$.
Fortunately, we know that we can compute $S_I^{1/e_j} = S^{1/(e_I e_j)}$ using <a href="#shamirs-trick">Shamir’s trick</a> on $S_I = S^{1/e_I}$ and $S^{1/e_j}$!
(This takes $O(\ell \vert I \vert)\ \Gho$ operations which is faster when $|I|$ is much smaller than $n$.)</p>
<p class="info">This technique can also be generalized to updating $\pi_I$ after many positions $j\in J$ changed by $\delta_j$.
(Assume without loss of generality that $J\cap I =\varnothing$ or let $J \leftarrow J \setminus I$ otherwise.)
This either requires all $S_I^{1/e_j}$ update keys or the <em>aggregated update key</em> \(S_{I\cup J} = S^{1/(e_I e_J)} = S^{\prod_{k\in I\cup J} 1/e_k}\) as auxiliary information.
In this last case, all the \((S_I^{1/e_j})_{j\in J} \leftarrow \rootfactor(S_{I\cup J}, (e_j)_{j\in J})\) can be computed in $O(\ell \vert J \vert \log \vert J \vert)\ \Gho$ operations from the aggregated update key.
Then, \(\Lambda_I' = \Lambda_I \cdot \prod_{j\in J} (S_I^{1/e_j})^{\delta_j}\).</p>
<h3 id="case-3-adding-new-positions">Case 3: Adding new positions</h3>
<p>Third, suppose a new position $v_{n+1}$ was added to the vector.
Then the proof update proceeds similar to the <a href="#adding-new-positions-to-the-vector">digest update</a> for adding new positions:
\begin{align}
S_I’ &= S_I^{e_{n+1}}\\<br />
\Lambda_I’ &= \Lambda_I^{e_{n+1}} S_I^{v_{n+1}} = \dots = \prod_{i\in [n+1] \setminus I} (S_I’^{1/e_i})^{v_i}
\end{align}</p>
<h2 id="disaggregating-proofs">Disaggregating proofs</h2>
<p>Suppose we want to <strong>disaggregate</strong> a proof $\pi_I=(S_I,\Lambda_I)$ for $\vect{v}_I$ into a proof $\pi_K=(S_K,\Lambda_K)$ for $\vect{v}_K$, where $K\subset I$.</p>
<p>In other words, we want to go from $d(\vect{v}\setminus I)$ to $d(\vect{v}\setminus K)$.
Since $K\subset I$ we can write $K = I\setminus \Delta$ for some set $\Delta$ of indices.
<!-- Thus, slightly abusing notation, $(\vect{v} \setminus K) = (\vect{v} \setminus (I \setminus \Delta)) = (\vect{v} \setminus I \cup \Delta)$ -->
Thus, all we need to do is add back the elements from $\Delta$ to $\pi_I$.</p>
<p>So, for each $i\in \Delta$, we simply add back \((i,v_i)\) to \(\pi_I\) (as explained <a href="#updating-digest">here</a>) and obtain \(\pi_K\).
This takes $O(\ell\vert\Delta\vert)$ $\Gho$ operations.</p>
<h2 id="aggregating-proofs">Aggregating proofs</h2>
<p>To aggregate two subvector proofs for \(\vect{v}_I\) and \(\vect{v}_J\), things are bit more involved.
Let \(\pi_I = (S_I,\Lambda_I)\) and \(\pi_J = (S_J,\Lambda_J)\) denote the two proofs.
Assume that \(I\cap J = \varnothing\) and if not, just set \(J = J\setminus I\) and disaggregate \(\pi_J\) into \(\pi_{J\setminus I}\).
(Can also set \(I=I\setminus J\) and disaggregate \(\pi_I\) instead.)</p>
<p>Denote the aggregated proof by \(\pi_{I\cup J} = (S_{I \cup J}, \Lambda_{I \cup J})\).
We can easily compute the first part of the proof as:</p>
\[S_{I\cup J} = \shamirtrick(S_I, S_J, e_I, e_J) = S^{1/(e_I e_J)}\]
<p>For the second part, focusing on $\Lambda_I$, recall that:
\begin{align}
\Lambda_I &= \prod_{i\in [n]\setminus I} (S_I^{1/e_i})^{v_i}
\end{align}
We will tweak it as \(\Lambda_I^*\) by removing all elements $v_j, j\in J$ from \(\Lambda_I\):
\begin{align}
\Lambda_I^* &= \Lambda_I / \prod_{j\in J} (S_I^{1/e_j})^{v_j}\\<br />
&= \prod_{i\in [n]\setminus (I\cup J)} (S_I^{1/e_i})^{v_i}
\end{align}
But for this, we need to compute \(S_I^{1/e_j},\forall j\in J\) via $\rootfactor$:
\begin{align}
(S_I^{1/e_j})_{j\in J} \leftarrow \rootfactor(S_{I\cup J}, (e_j)_{j\in J})
\end{align}
In a similar, fashion, we can compute
\begin{align}
\Lambda_J^* &= \prod_{i\in [n]\setminus (I\cup J)} (S_J^{1/e_i})^{v_i}
\end{align}
Next, observe that \(\Lambda_I^*\) and \(\Lambda_J^*\) can be rewritten as:
\begin{align}
\Lambda_I^* &= \left(\prod_{i\in [n]\setminus (I\cup J)} (S^{1/e_i})^{v_i}\right)^{1/e_I}\\<br />
\Lambda_J^* &= \left(\prod_{i\in [n]\setminus (I\cup J)} (S^{1/e_i})^{v_i}\right)^{1/e_J}
\end{align}
Thus, if we apply a Shamir trick on them, we can obtain $\Lambda_{I\cup J}$:
\begin{align}
\Lambda_{I\cup J} &= \shamirtrick(\Lambda_I^*, \Lambda_J^*, e_I, e_J)\\<br />
&= \left(\prod_{i\in [n]\setminus (I\cup J)} (S^{1/e_i})^{v_i}\right)^{1/(e_I e_J)}\\<br />
&= \prod_{i\in [n]\setminus (I\cup J)} (S_{I\cup J}^{1/e_i})^{v_i}
\end{align}</p>
<p class="info">Overall, if $b=\max(\vert I\vert, \vert J \vert)$, this requires Shamir tricks of size $b$ and $\rootfactor$’s of size $b$, so it takes $O(\ell b\log{b})\ \Gho$ operations.</p>
<h2 id="precomputing-all-proofs-via-disaggregation">Precomputing all proofs via disaggregation</h2>
<p>Campanelli et al.<sup id="fnref:CFGplus20e:1" role="doc-noteref"><a href="#fn:CFGplus20e" class="footnote">3</a></sup> explain how to compute all proofs fast via disaggregation.
The idea is to start with a proof for the whole vector $\vect{v}$ and then disaggregate it into a proof for the left half of the vector and the right half.
Then, one repeats recursively on these two halves.
This determines a tree of subvector proofs, where the leaves store the proofs for the individual vector and each internal node stores an aggregation of its children’s proofs (with the root storing the proof for the full vector).</p>
<!--
NOTE: This was the LaTeX for the tree.
Just be sure to add \usepackage{forest} in the preamble in OS X's "LaTeX it!" if you redo this.
\begin{figure*}[t]
{
\normalsize
\begin{center}
\begin{forest}
for tree={
% fit=band,% spaces the tree out a little to avoid collisions
% fit=tight,% spaces the tree out less
% fit=rectangle,
inner sep=4,
}
[{$\pi_{[1,8]},\{v_1,\dots,v_8\}$}
[{$\pi_{[1,4]},\{v_1,\dots,v_4\}$}
[{$\pi_{[1,2]},\{v_1,v_2\}$}
[{$\pi_1,\{v_1\}$}
[, no edge, tier=odd ]
]
[{$\pi_2,\{v_2\}$}
, tier=odd
]
]
[{$\pi_{[3,4]},\{v_3,v_4\}$}
[{$\pi_3,\{v_3\}$}
[, no edge, tier=odd ]
]
[{$\pi_4,\{v_4\}$}
, tier=odd
]
]
]
[{$\pi_{[5,8]},\{v_5,\dots,v_8\}$}
[{$\pi_{[5,6]},\{v_5,v_6\}$}
[{$\pi_5,\{v_5\}$}
[, no edge, tier=odd ]
]
[{$\pi_6,\{v_6\}$}
, tier=odd
]
]
[{$\pi_{[7,8]},\{v_7,v_8\}$}
[{$\pi_7,\{v_7\}$}
[, no edge, tier=odd ]
]
[{$\pi_8,\{v_8\}$}
, tier=odd
]
]
]
]
\end{forest}
\end{center}
}
\end{figure*}
-->
<p>Here’s an example for $n=8$:</p>
<div align="center"><img style="width:95%" src="/pictures/cf-proof-precomp.png" /></div>
<p>Looked at it differently, clearly one can compute this tree via proof aggregation, which we discuss <a href="#aggregating-proofs">next</a>, by starting with the leaves storing individual proofs for each position $i$ and aggregating up the tree.
The key observation is that one can also compute this tree by starting with the root, which stores the proof for the full vector (i.e., the empty digest), <em>disaggregating</em> this proof into two halves, and recursing on these two halves.</p>
<p class="info">This very much resembles the recursion tree used to implement $\rootfactor$ efficiently (see <a href="/2020/11/24/RSA-accumulators.html#precomputing-all-membership-witnesses-fast">here</a>).
Indeed, if you think about it, this tree implicitly computes the $\rootfactor$ tree too, since each CF proof for $v_i$ has an RSA membership witness for $i$ in it.</p>
<p>Note that Campanelli et al.<sup id="fnref:CFGplus20e:2" role="doc-noteref"><a href="#fn:CFGplus20e" class="footnote">3</a></sup> claim $O(\ell n\log^2{n})\ \Gho$ operations for this algorithm, but the faster disaggregation explained above actually gives $O(\ell n\log{n})$.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:CF13e" role="doc-endnote">
<p><strong>Vector Commitments and their Applications</strong>, by Dario Catalano and Dario Fiore, <em>in Cryptology ePrint Archive, Report 2011/495</em>, 2011, <a href="https://eprint.iacr.org/2011/495">[URL]</a> <a href="#fnref:CF13e" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:LM18" role="doc-endnote">
<p><strong>Subvector Commitments with Application to Succinct Arguments</strong>, by Russell W.F. Lai and Giulio Malavolta, <em>in Cryptology ePrint Archive, Report 2018/705</em>, 2018, <a href="https://eprint.iacr.org/2018/705">[URL]</a> <a href="#fnref:LM18" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:CFGplus20e" role="doc-endnote">
<p><strong>Vector Commitment Techniques and Applications to Verifiable Decentralized Storage</strong>, by Matteo Campanelli and Dario Fiore and Nicola Greco and Dimitris Kolonelos and Luca Nizzardo, 2020, <a href="https://eprint.iacr.org/2020/149">[URL]</a> <a href="#fnref:CFGplus20e" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:CFGplus20e:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a> <a href="#fnref:CFGplus20e:2" class="reversefootnote" role="doc-backlink">↩<sup>3</sup></a></p>
</li>
</ol>
</div>Alin TomescuA vector commitment (VC) scheme allows a prover with access to a vector $\mathbf{v} = [ v_1, \dots, v_n ]$ to convince any verifier that position $i$ in $\mathbf{v}$ stores $v_i$ for any index $i\in[n]$. Importantly, verifers only store a succinct digest of the vector (e.g., a 32-byte hash) rather than the full vector $\mathbf{v}$.RSA Accumulators2020-11-24T00:00:00+00:002020-11-24T00:00:00+00:00https://alinush.github.io//2020/11/24/RSA-accumulators<p>An <strong>RSA accumulator</strong> is an <em>authenticated set</em> built from cryptographic assumptions in hidden-order groups such as $\mathbb{Z}_N^*$.
RSA accumulators enable a <strong>prover</strong>, who stores the full set, to convince any <strong>verifier</strong>, who only stores a succinct <strong>digest</strong> of the set, of various set relations such as (non)membership, subset or disjointness.
For example, the prover can prove membership of elements in the set to verifiers who have the digest.</p>
<!--more-->
<p hidden="">$$
\def\Adv{\mathcal{A}}
\def\Badv{\mathcal{B}}
\def\GenGho{\mathsf{GenGroup}_?}
\def\Ghosz{|\Gho|}
\def\Ghoid{1_{\Gho}}
\def\primes{\mathsf{Primes}}
\def\QRn{\mathsf{QR}_N}
\def\multirootexp{\mathsf{MultiRootExp}}
\def\rootfactor{\mathsf{RootFactor}}
\def\vect#1{\mathbf{#1}}
$$</p>
<p>The <em>digest</em> of the set is often referred to as the <em>accumulator</em> of the set.
<!-- TODO: Link to RSA assumptions post --></p>
<p>RSA accumulators were introduced by <em>Benaloh and de Mare</em><sup id="fnref:Bd93" role="doc-noteref"><a href="#fn:Bd93" class="footnote">1</a></sup> and later extended by <em>Li et al.</em><sup id="fnref:LLX07" role="doc-noteref"><a href="#fn:LLX07" class="footnote">2</a></sup> with non-mmebership proofs.
Recently, Boneh et al.<sup id="fnref:BBF18" role="doc-noteref"><a href="#fn:BBF18" class="footnote">3</a></sup> extended RSA accumulators with many new features.</p>
<p class="warning">This post is not a full treatment of RSA accumulators, but will be extended over time.</p>
<h2 id="digest-or-accumulator-of-a-set">Digest (or accumulator) of a set</h2>
<p>Let $H$ be a collision-resistant hash function that maps its input to prime numbers.
Let $g$ be the generator of a hidden-order group $\Gho$ where the Strong RSA problem is hard.
<!-- TODO: reference assumptions -->
Let $T = \{b_1, b_2, \dots, b_n\}$ and let $e_i = H(b_i)$ be the <em>prime representative</em> of $b_i$.</p>
<p>The accumulator of $T$ is:</p>
\[a = g^{\prod_{i\in[n]} e_i}\]
<p>Note that this can be computed in $O(n)$ exponentiations in $\Gho$.</p>
<h2 id="membership-witnesses">Membership witnesses</h2>
<p>To prove that $b_i$ is in the accumulator, a <em>membership witness</em> can be computed:</p>
\[w_i = g^{\prod_{j\in[n]\setminus\{i\}} e_j} = a^{1/e_i}\]
<p>Note that the witness is simply the accumulator of the set $T \setminus \{b_i\}$ and is just an $e_i$th root of $a$!</p>
<p class="warning">Unfortunately, computing this root cannot be done using an exponentiation by $1/e_i$ because $e_i$ cannot be inverted without knowing the order of the group $\Gho$.
Instead, computing $w_i$ has to be done by exponentiating $g$ by the $n-1$ different $e_i$’s and thus takes $O(n)$ time, which can be slow.
We explain <a href="#precomputing-all-membership-witnesses-fast">below</a> how this can be done in $O(\log{n})$ amortized time per witness.</p>
<p>To verify the witness against the accumulator $a$, one checks if:</p>
\[a \stackrel{?}{=} w_i^{e_i}\]
<p>In other words, one simply “adds back” $b_i$ to the set accumulated in $w_i$ and checks if the result equals $a$.
This takes one exponentiation in $\Gho$.</p>
<p>Membership witnesses can be easily generalized into <em>batch membership witnesses</em>.
For example, if one wants to prove that all $(b_i)_{i\in I}$ are in the accumulator, they can compute:</p>
\[w_I = g^{\prod_{i\in[n]\setminus I} e_i} = a^{1/ \prod_{i\in I} e_i}\]
<p>The verification proceeds analogously but takes $O(\vert I \vert)$ exponentiations:</p>
\[a \stackrel{?}{=} w_i^{\prod_{i\in I}e_i}\]
<h2 id="precomputing-all-membership-witnesses-fast">Precomputing all membership witnesses fast</h2>
<p>Computing all $n$ membership witnesses naively takes $O(n^2)$ exponentiations in $\Gho$, which does not scale well.
Fortunately, <em>Sander et al.</em><sup id="fnref:SSY01" role="doc-noteref"><a href="#fn:SSY01" class="footnote">4</a></sup> give a divide-and-conquer approach for computing all witnesses which takes $O(n\log{n})$ exponentiations in $\Gho$.</p>
<p>The key observation is that half of the witnesses require computing $g^{\prod_{i\in[1,n/2]} e_i}$ while the other half require computing $g^{\prod_{i\in[n/2+1,n]} e_i}$.
If done naively, these computations would be repeated many times unnecessarily.
But one could compute the witnesses recursively in a tree-like fashion as follows ($n=8$ in this example):</p>
<div align="center"><img style="width:100%" src="/pictures/rsa-membwit-precomp.png" /></div>
<p class="info"><strong>Intuition:</strong>
You can think of each node as (1) a set of elements and (2) its <em>batch membership witness</em> w.r.t. the accumulator $a$.
Then, this algorithm simply splits the set into two halves and disaggregates the witness into witnesses for the two halves.
This repeats until witnesses for individual elements are obtained at the bottom.</p>
<p>In other words, this algorithm computes every $e_i$th root of $a$ (i.e., $g^{e_1 \dots e_{i-1} e_{i+1} \dots e_{n}}$) for all $i\in[n]$.
The time to compute a tree of size $n$ leaves can be defined recursively as $T(n) = 2T(n/2) + O(n)$ because the amount of work done at the root of a size-$n$ subtree is $O(n)$ exponentiations plus the time to recurse on its two children.
This gives $T(n) = O(n\log{n})$ exponentiations.</p>
<p>Boneh et al.<sup id="fnref:BBF18:1" role="doc-noteref"><a href="#fn:BBF18" class="footnote">3</a></sup> baptised this algorithm as \(\rootfactor\).
Specifically,</p>
\[\rootfactor(g, (e_i)_{i\in[n]}) = (w_i)_{i\in[n]} = (a^{1/e_i})_{i\in[n]} = \left((g^{\prod_{j\in[n]} e_j})^{1/e_i}\right)_{i\in[n]}\]
<!-- **TODO:** Future post: update membership, aggregate membership, non-membership witnesses + update, subset relations, disjointness relations -->
<h2 id="more-to-come">More to come</h2>
<p>There is so much more to be said about RSA accumulators!
Hopefully, we will update this post over time with:</p>
<ul>
<li>Accumulator updates</li>
<li>Membership witness updates</li>
<li>Cross-incremental (dis)aggregation of membership witnesses<sup id="fnref:TXN20e" role="doc-noteref"><a href="#fn:TXN20e" class="footnote">5</a></sup></li>
<li>Non-membership witnesses
<ul>
<li>Updates</li>
<li>Aggregation</li>
</ul>
</li>
<li>Disjointness witnesses<sup id="fnref:Tome20" role="doc-noteref"><a href="#fn:Tome20" class="footnote">6</a></sup></li>
<li>Subset witnesses<sup id="fnref:Tome20:1" role="doc-noteref"><a href="#fn:Tome20" class="footnote">6</a></sup></li>
</ul>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:Bd93" role="doc-endnote">
<p><strong>One-Way Accumulators: A Decentralized Alternative to Digital Signatures</strong>, by Benaloh, Josh and de Mare, Michael, <em>in EUROCRYPT ‘93</em>, 1994 <a href="#fnref:Bd93" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:LLX07" role="doc-endnote">
<p><strong>Universal Accumulators with Efficient Nonmembership Proofs</strong>, by Li, Jiangtao and Li, Ninghui and Xue, Rui, <em>in Applied Cryptography and Network Security</em>, 2007 <a href="#fnref:LLX07" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:BBF18" role="doc-endnote">
<p><strong>Batching Techniques for Accumulators with Applications to IOPs and Stateless Blockchains</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/1188</em>, 2018, <a href="https://eprint.iacr.org/2018/1188">[URL]</a> <a href="#fnref:BBF18" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:BBF18:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:SSY01" role="doc-endnote">
<p><strong>Blind, Auditable Membership Proofs</strong>, by Sander, Tomas and Ta-Shma, Amnon and Yung, Moti, <em>in Financial Cryptography</em>, 2001 <a href="#fnref:SSY01" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:TXN20e" role="doc-endnote">
<p><strong>Authenticated Dictionaries with Cross-Incremental Proof (Dis)aggregation</strong>, by Alin Tomescu and Yu Xia and Zachary Newman, <em>in Cryptology ePrint Archive, Report 2020/1239</em>, 2020, <a href="https://eprint.iacr.org/2020/1239">[URL]</a> <a href="#fnref:TXN20e" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Tome20" role="doc-endnote">
<p><strong>How to Keep a Secret and Share a Public Key (Using Polynomial Commitments)</strong>, by Tomescu, Alin, 2020, <a href="https://alinush.github.io/papers/phd-thesis-mit2020.pdf">[URL]</a> <a href="#fnref:Tome20" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:Tome20:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
</ol>
</div>Alin TomescuAn RSA accumulator is an authenticated set built from cryptographic assumptions in hidden-order groups such as $\mathbb{Z}_N^*$. RSA accumulators enable a prover, who stores the full set, to convince any verifier, who only stores a succinct digest of the set, of various set relations such as (non)membership, subset or disjointness. For example, the prover can prove membership of elements in the set to verifiers who have the digest.Linear Diophantine Equations2020-11-23T00:00:00+00:002020-11-23T00:00:00+00:00https://alinush.github.io//2020/11/23/Linear-Diophantine-Equations<p>Equations of the form $\sum_i a_i x_i = 0$ where the $x_i$’s are <em>integer</em> unknowns are called <strong>linear Diophantine equations.</strong>
Their integer solutions can be computed using greatest common denominator (GCD) tricks.
In this post, we go over a few basic types of such equations and their integer solutions.
<!--more--></p>
<p hidden="">$$
\def\lcm{\ \text{lcm}}
$$</p>
<h2 id="axby--t">$ax+by = t$</h2>
<details>
<summary><b>Theorem:</b> This equation has an integer solution $\Leftrightarrow \gcd(a,b) \mathrel\vert t$.
</summary>
<p style="margin-left: .3em; border-left: .15em solid black; padding-left: .5em;">
<em>Proof</em> (“$\Rightarrow$”): Assume an integer solution $(x_0, y_0)$ exists when $\gcd(a,b) \nmid t$.
Since $\gcd(a,b)$ divides both $a$ and $b$, it divides any linear combination of them, including $ax_0 + by_0 = t$, which implies it divides $t$.
Contradiction.
<br /><br />
<em>Proof</em> (“$\Rightarrow$”):
If $\gcd(a,b) = 1$, then an integer solution $x_0,y_0$ can be obtained using the Extended Eucliden algorithm, which finds $(u,v)$ such that $au + bv = \gcd(a,b)$.
The solution is:
\begin{align}
x_0 &= u\cdot t / \gcd(a,b)\\\
y_0 &= v\cdot t / \gcd(a,b)
\end{align}
This is because $ax_0 + ay_0 = (au + bv) \frac{t}{\gcd(a,b)} = \gcd(a,b) \frac{t}{\gcd(a,b)} = t$.
</p>
</details>
<details>
<summary><b>Theorem:</b> When $\gcd(a,b) \mathrel\vert t$, if $(x_0, y_0)$ is an integer solution to such an equation, then all integer solutions can be characterized as:
\begin{align}
x &= x_0 - k b/\gcd(a,b)\\\
y &= y_0 + k a/\gcd(a,b)
\end{align}
Here, $k$ is an arbitrary integer and either $a\ne 0$ or $b\ne 0$ (since otherwise $\gcd(a,b) = 0$ and $\gcd(a,b)\mathrel\vert t$ implies $t=0$, which means all integers $(x,y)$ are solutions).
</summary>
<p style="margin-left: .3em; border-left: .15em solid black; padding-left: .5em;">
<em>Proof:</em> First, one can easily verify that the proposed $(x,y)$ are indeed solutions that satisfy $ax+by=t$:
\begin{align}
ax + by &= ax_0 + kab / \gcd(a,b) + by_0 - kab/gcd(a,b)\\
&= ax_0 + by_0 = t
\end{align}
The more difficult part is to argue that every solution has this form!
Assume $a\ne 0$ since the other $b\ne 0$ case is symmetric.
Assume $(x,y)$ to be an integer solution and note that, since $(x_0, y_0)$ is a solution, this means:
\begin{align}
a(x-x_0) + b(y - y_0) &= 0\Leftrightarrow\\
a(x - x_0) &= b(y_0 - y)\Leftrightarrow\\
\frac{a}{\gcd(a,b)}(x-x_0) &= \frac{b}{\gcd(a,b)} (y_0 - y)
\end{align}
This implies $\frac{a}{\gcd(a,b)} \mathrel\vert \frac{b}{\gcd(a,b)} (y - y_0)$.
Since $\gcd(\frac{a}{\gcd(a,b)}, \frac{b}{\gcd(a,b)}) = 1$<sup id="fnref:Heff03" role="doc-noteref"><a href="#fn:Heff03" class="footnote">1</a></sup>, this means $\frac{a}{gcd(a,b)} \mathrel\vert (y-y_0)$, which means $\exists k$ such that:
\begin{align}
a/\gcd(a,b) \cdot k &= (y-y_0)\Leftrightarrow\\
y &= y_0 + k a / \gcd(a,b)
\end{align}
Next, substitute $y$ in $a(x-x_0) = b(y_0 - y)$, to get:
\begin{align}
a(x-x_0) &= b(y_0 - y_0 - k a / \gcd(a,b))\Leftrightarrow\\
x - x_0 &= (- b k a / \gcd(a,b)) / a\\
&= x_0 - k b / \gcd(a,b)
\end{align}
Note that since $a\ne 0$, we are allowed to divide by $a$ above.
</p>
</details>
<h3 id="axby--0">$ax+by = 0$</h3>
<p>In this case, since $x_0 = 0$ and $y_0 = 0$ is one integer solution, all integer solutions are of the form:
\begin{align}
x &= -kb/\gcd(a,b)\\<br />
y &= ka/\gcd(a,b)
\end{align}</p>
<h2 id="axbycz--t">$ax+by+cz = t$</h2>
<details>
<summary><b>Theorem:</b> This equation has an integer solution $\Leftrightarrow \gcd(a,b,c) \mathrel\vert t$.
</summary>
<p style="margin-left: .3em; border-left: .15em solid black; padding-left: .5em;">
<em>Proof</em> (“$\Rightarrow$”):
Proceeds analogously to the $ax+by = t$ case <a href="#axby--t">before</a>.
<br /><br />
<em>Proof</em> (“$\Leftarrow$”):
We will generalize the proof from the $ax+by = t$ case <a href="#axby--t">before</a>.
<br /><br />
$\gcd(a,b,c) \mathrel\vert t \Rightarrow \gcd(a, \gcd(b, c)) \mathrel\vert t \Rightarrow ax + \gcd(b,c)w = t$ has an integer solution $(x_0, w_0)$.
<br /><br />
Let $\gcd(b,c) = d$.
We know $\exists (y_0, z_0)$ such that:
\begin{align}
b y_0 + c z_0 = d
\end{align}
Replacing in the previous equation, we have:
\begin{align}
ax_0 + \gcd(b,c)w_0 &= t\Leftrightarrow\\
ax_0 + dw_0 &= t\Leftrightarrow\\
ax_0 + (b y_0 + c z_0) w_0 &= t\Leftrightarrow\\
ax_0 + b w_0 y_0 + c w_0 z_0 &= t
\end{align}
Thus, an integer solution is $(x_0, y_0 w_0, z_0 w_0)$.
</p>
</details>
<p><strong>Theorem</strong> <em>(from Sec 6.2 in <sup id="fnref:Cohe07" role="doc-noteref"><a href="#fn:Cohe07" class="footnote">2</a></sup>)</em>: In this case, if $(x_0, y_0, z_0)$ is one integer solution, then all integer solutions are of the form:
\begin{align}
x &= x_0 + m b / \gcd(a,b) - \ell c / \gcd(a,c)\\<br />
y &= y_0 + k c / \gcd(b,c) - m a / \gcd(a, b)\\<br />
z &= z_0 + \ell a / \gcd(a, c) - k b / \gcd(b, c)
\end{align}
Here, $m,\ell,k$ are integers and at least one of $a,b$ or $c$ are $\ne 0$.</p>
<!--
_Proof:_ It is easy to check that proposed integers $x,y,z$ are indeed valid solutions.
The more difficult part is to argue that every $(x,y,z)$ solution has this form!
Assume $a\ne 0$ since the other $b\ne 0$ case is symmetric.
Assume $(x,y,z)$ to be an integer solution and note that, since $(x_0, y_0, z_0)$ is a solution, this means:
\begin{align}
a(x-x_0) + b(y - y_0) + c(z-z_0) &= 0\Leftrightarrow\\\\\
a(x - x_0) &= b(y_0 - y) + c(z_0 - z)\Leftrightarrow\\\\\
\end{align}
**TODO: continue**
-->
<h3 id="axbycz--0">$ax+by+cz = 0$</h3>
<p>In this case, all integer solutions are of the form:
\begin{align}
x &= m b / \gcd(a,b) - \ell c / \gcd(a,c)\\<br />
y &= k c / \gcd(b,c) - m a / \gcd(a, b)\\<br />
z &= \ell a / \gcd(a, c) - k b / \gcd(b, c)
\end{align}
Here, $m,\ell,k$ are arbitrary integers.</p>
<!-- TODO: proof
See page 10 here: https://people.math.sc.edu/howard/Classes/580f/hw5.pdf
See https://math.stackexchange.com/questions/3325185/find-all-the-integral-solutions-to-the-equation-323x391y437z-10473/3327153#3327153
-->
<p>This could also be simplified in terms of the <em>lowest common multiple (LCM)</em>, since $a b = \gcd(a,b) \lcm(a,b)\Rightarrow b / \gcd(a,b) = \lcm(a,b) / a$:
\begin{align}
x &= m \lcm(a,b) / a - \ell \lcm(a,c) / a = \frac{m\lcm(a,b) - \ell\lcm(a,c)}{a}\\<br />
y &= k \lcm(b,c) / b - m \lcm(a, b) / b = \frac{k\lcm(b,c) - m\lcm(a,b)}{b}\\<br />
z &= \ell \lcm(a, c) / c - k \lcm(b, c) / c = \frac{\ell\lcm(a,c) - k\lcm(b,c)}{c}
\end{align}</p>
<h4 id="an-example">An example</h4>
<p>In a Catalano-Fiore vector commitment (VC)<sup id="fnref:CF13e" role="doc-noteref"><a href="#fn:CF13e" class="footnote">3</a></sup> of size $n=3$, collision resistance is implied by the fact that the following equation with $(\ell+1)$-bit primes $e_1,e_2,e_3$ does not have any $\ell$-bit integer solutions:</p>
\[e_2 e_3 v_1 + e_1 e_3 v_2 + e_1 e_2 v_3 = 0\]
<p>The only integer solutions given by the formula above are at least $\ell+1$ bit wide:
\begin{align}
%x &= \frac{m \lcm(e_2 e_3, e_1 e_3) - \ell\lcm(e_2 e_3, e_1 e_2)}{e_2 e_3}\\<br />
% &= \frac{m e_1 e_2 e_3 - \ell e_1 e_2 e_3}{e_2 e_3} = (m-\ell)e_1\\<br />
x &= m (e_1 e_3) / \gcd(e_2 e_3, e_1 e_3) - \ell (e_1 e_2) / \gcd(e_2 e_3, e_1 e_2) =\\<br />
&= m (e_1 e_3) / e_3 - \ell (e_1 e_2) / e_2 = m’ e_1\ (\text{where}\ m’=m-\ell)\\<br />
y &= k’ e_2\\<br />
z &= \ell’ e_3
\end{align}</p>
<h2 id="sum_iinn-a_i-x_i--t">$\sum_{i\in[n]} a_i x_i = t$</h2>
<p><strong>Theorem:</strong> This equation has an integer solution $\Leftrightarrow \gcd(a_1, \dots, a_n) \mathrel\vert t$.</p>
<p><em>Proof:</em> The “$\Rightarrow$” direction proceeds as <a href="#axbycz--t">before</a>.</p>
<p>The “$\Leftarrow$” direction proceeds by induction.
The statement $P(n)$ being proved by induction is:</p>
\[\forall a_i\in \mathbb{Z}, \gcd(a_1, \dots, a_n) \mathrel\vert t\Rightarrow \sum_{i\in[n]} a_i x_i = t\ \text{has an integer solution.}\]
<p>We prove $P(n)$ is true for all $n \ge 2$.
First, $P(2)$ is clearly true as we’ve shown <a href="#axby--t">before</a>.
Second, we must show $P(n) \Rightarrow P(n + 1)$.
This we do analogously to the proof for $n=3$ from <a href="#axbycz--t">before</a>.
Let $d = \gcd(a_1, \dots, a_n)$.
We know that $\gcd(a_1, \dots, a_{n+1}) = \gcd(d, a_{n+1})$.
Since $d \mathrel| t$ (because $P(n)$ is true), this implies $\exists$ integers $s_0, w_0$ such that:
\begin{align}
d s_0 + a_{n+1} w_0 = t \Leftrightarrow
\end{align}</p>
<p>Since $P(n)$ is true, we know there exist integers $s_1,\dots, s_n$ such that:
\begin{align}
\sum_{i\in[n]} a_i s_i = d
\end{align}</p>
<p>Replacing $d$ in the previous equation, we get:
\begin{align}
d s_0 + a_{n+1} w_0 &= t \Leftrightarrow\\<br />
\left(\sum_{i\in[n]} a_i s_i\right) s_0 + a_{n+1} w_0 &= t
\end{align}
Thus, a solution can be found by setting $x_i = s_0 s_i,\forall i\in[n]$ and $x_{n+1} = w_0$.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:Heff03" role="doc-endnote">
<p><strong>Elementary Number Theory</strong>, by Jim Hefferson and W. Edwin Clark, 2004, <a href="http://joshua.smcvt.edu/numbertheory/book.pdf">[URL]</a> <a href="#fnref:Heff03" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Cohe07" role="doc-endnote">
<p><strong>Number Theory: Volume I: Tools and Diophantine Equations</strong>, by Cohen, H., 2007, <a href="https://books.google.com/books?id=8zC8VPQV8psC">[URL]</a> <a href="#fnref:Cohe07" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:CF13e" role="doc-endnote">
<p><strong>Vector Commitments and their Applications</strong>, by Dario Catalano and Dario Fiore, <em>in Cryptology ePrint Archive, Report 2011/495</em>, 2011, <a href="https://eprint.iacr.org/2011/495">[URL]</a> <a href="#fnref:CF13e" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Alin TomescuEquations of the form $\sum_i a_i x_i = 0$ where the $x_i$’s are integer unknowns are called linear Diophantine equations. Their integer solutions can be computed using greatest common denominator (GCD) tricks. In this post, we go over a few basic types of such equations and their integer solutions.Cryptographic Assumptions in Hidden-Order Groups2020-11-05T00:00:00+00:002020-11-05T00:00:00+00:00https://alinush.github.io//2020/11/05/cryptographic-assumptions-in-hidden-order-groups<p>In this post, we summarize some of the cryptographic hardness assumptions used in <strong>hidden-order groups</strong>.</p>
<!--more-->
<p hidden="">$$
\def\Adv{\mathcal{A}}
\def\Badv{\mathcal{B}}
\def\GenGho{\mathsf{GenGroup}_?}
\def\Ghosz{|\Gho|}
\def\Ghoid{1_{\Gho}}
\def\primes#1{\mathsf{Primes}_{#1}}
\def\QRn{\mathsf{QR}_N}
$$</p>
<h2 id="terminology-and-notation">Terminology and notation</h2>
<p>We try to describe these assumptions in terms of a generic hidden-order group $\Gho$ of order $\Ghosz$.
We denote the identity element in such a group by $\Ghoid$.</p>
<p>Sometimes, we specifically refer to the RSA group $\Gho=\ZNs$.
Specifically, let $N = pq$ be the product of two sufficiently-large prime integers $p, q$.
Then, \(\ZNs = \{0 < a < N \mathrel| \gcd(a, N) = 1\}\) is the multiplicative group of integers co-prime with $N$.
Recall that the size or <em>order</em> of this group is given by the totient function: $|\ZNs| = \phi(N) = (p-1)(q-1)$.</p>
<p>Other times we might also refer to the <em>class group of imaginary quadratic orders</em> introduced by Buchmann and Williams<sup id="fnref:BW88" role="doc-noteref"><a href="#fn:BW88" class="footnote">1</a></sup>.</p>
<p class="warning">When we say <em>“Assumption $A$ implies assumption $B$”</em> this means that <em>“for $A$ to hold, $B$ must hold”</em>.
Or, put differently, this means that <em>“$A$ can be broken given an algorithm that breaks $B$”</em>.
For example, to say that <em>“the Strong RSA assumption implies the order assumption (OA)”</em> is the same thing as saying <em>“the Strong RSA problem reduces to the order problem”</em>, which is the same thing as saying <em>“given an algorithm for solving the order problem, one can solve the Strong RSA problem.”</em></p>
<p>All of our probability clauses $\Pr[\dots]=\negl(\lambda)$ below would more formally be stated as “for all <em>polynomial probabilistic time (PPT)</em> adversaries $\Adv$, there exists a negligible function $\negl(\cdot)$ such that $\Pr[\dots] = \negl(\lambda)$.</p>
<!--
**TODO:**
- Define $\QRn$
+ Include Quadratic Residuosity Assumption (QRA) (see [here](https://en.wikipedia.org/wiki/Quadratic_residuosity_problem) and [here](https://crypto.stanford.edu/pbc/notes/numbertheory/qr.html))
- Define **generic group model (GGM)** and **generic adversary**[^DK02]
- _Blum integers_ as defined by Hastad et al.[^HSS93]
-->
<h2 id="assumptions">Assumptions</h2>
<h3 id="factoring">Factoring</h3>
\[\Pr
\begin{bmatrix}
p, q \xleftarrow{\$}\primes{\poly(\lambda)},\\
N = pq : \\
(p,q) \leftarrow \Adv(N) \\
\end{bmatrix} \leq \negl(\lambda)\]
<!--
**TODO:**
- Eh, I'm actually not sure about how to precisely formalize the length of $p$ and $q$.
- Factoring $n$ knowing $e$ and $d$[^Bone99] in more detail via [StackExchange explantion](https://crypto.stackexchange.com/questions/78330/factoring-n-in-rsa-knowing-e-and-d) and [other SE answer](https://crypto.stackexchange.com/a/43228/21296)
+ Implementation in [cryptopp](https://github.com/weidai11/cryptopp/blob/CRYPTOPP_8_2_0/rsa.cpp#L160)
-->
<h3 id="discrete-logarithm">Discrete logarithm</h3>
\[\Pr
\begin{bmatrix}
\Gho \leftarrow \GenGho(\lambda),\\
(g,h) \xleftarrow{\$} \Gho \times \Gho,\\
\ell \leftarrow \Adv(\Gho, g, h) : \\
g^{\ell} = h
\end{bmatrix} \leq \negl(\lambda)\]
<p><em>Bach</em><sup id="fnref:Bach84" role="doc-noteref"><a href="#fn:Bach84" class="footnote">2</a></sup> shows that factoring $N$ reduces to computing discrete logs in $\ZNs$.</p>
<!--
**TODO:**
- Is this a trivial reduction?
- What's the difference between restating the assumption as picking $\ell$ randomly and computing $h=g^{\ell}$. Most likely no difference except we'd have to ensure $\ell=O(\poly(\lambda))$
-->
<h3 id="the-rsa-assumption-rsa">The RSA assumption (RSA)</h3>
<p>Introduced by Rivest, Shamir and Adleman in their seminal 1978 paper on public-key cryptosystems<sup id="fnref:RSA78" role="doc-noteref"><a href="#fn:RSA78" class="footnote">3</a></sup>.</p>
<!--
**TODO:** Eh, need to bound the size of $\ell$ to be at most $\poly(\lambda)$.
But is there a different class of assumptions based on such restrictions?
-->
\[\Pr
\begin{bmatrix}
\Gho \leftarrow \GenGho(\lambda),\\
(g, \ell) \xleftarrow{\$} \Gho \times \Z\ \text{s.t.}\ \gcd(\ell, \Ghosz)=1\\
u \leftarrow \Adv(\Gho, g, \ell) : \\
g^{1/\ell} = u
\end{bmatrix} \leq \negl(\lambda)\]
<!--
**TODO:**
- Must $\ell$ be prime or not? BBHM02 assumes prime.
- Do we need to enforce the $\gcd$ check or does picking $\ell$ randomly suffice?
-->
<p class="warning">When $\Gho=\ZNs$, we cannot have $\ell=2$ because it would not be co-prime with $\phi(N) = (p-1)(q-1)$ and $f(x) = x^\ell = x^2$ would not be a permutation.
But if the subgroup of quadratic residues $\QRn$ is used, then $\ell = 2$ can be used (see <a href="https://crypto.stackexchange.com/a/65986/21296">here</a>).
So it is best not to restrict the definition above.</p>
<h3 id="strong-rsa-assumption">Strong RSA assumption</h3>
<p>This assumption says that, given a random $g\in \Gho$, it is hard to find <em>any root</em> of it: i.e., an integer $\ell$ in $\Z$ and a group element $u$ such that $g^{1/\ell} = u$.
Note that this generalizes the RSA assumption, which gives the adversary not only $g$, but also the root $\ell$ that should be computed.</p>
\[\Pr
\begin{bmatrix}
\Gho \leftarrow \GenGho(\lambda),\\
g\xleftarrow{\$} \Gho,\\
(u,\ell) \leftarrow \Adv(\Gho, g) : \\
g^{1/\ell} = u\ \text{and}\ \ell > 1
\end{bmatrix} \leq \negl(\lambda)\]
<p><strong>Strong RSA assumption implies RSA assumption:</strong>
If RSA assumption doesn’t hold then Strong RSA doesn’t either.
To see this, consider a Strong RSA adversary $\Adv$ that gets a random $g$ as input.
Since RSA doesn’t hold, there exists an RSA adversary $\Badv$ that given such a random $g$ and random $\ell\in \Z$ with $\gcd(\ell, \Ghosz) = 1$, outputs $u$ such that $g^{1/\ell} = u$.
Note that $\Adv$ can use $\Badv$ to break Strong RSA!
Specifically, $\Adv$ simply picks a random $\ell$ which, with overwhelming probability is co-prime with $\Ghosz$ (or else $\Adv$ can factor $N$ <!--**TODO: cite**-->).
Next, $\Adv$ calls $\Badv$ with $g$ and $\ell$, obtaining $u$ such that $g^{1/\ell} = u$ with non-negligible probability.
Finally, $\Adv$ simply outputs $(u,\ell)$, breaking Strong RSA with non-negligible probability.</p>
<!--
**TODO:**
- Again, what restrictions apply to $\ell$? Seems like $\ell > 2$ in RSA groups (though not sure how I would compute $u^{1/2}$ for random $u$.
- I believe [LM18] considers composite $\ell$ as a different assumption.
+ Yes, and they show Strong RSA implies their assumption.
-->
<h3 id="the-order-assumption-oa">The order assumption (OA)</h3>
<p>This assumption says that, given a random $g\in \Gho$, it is hard to find any multiple of its order: i.e., an integer $\ell$ such that $g^\ell = \Ghoid$
This is known as the <strong>order problem.</strong></p>
<p class="warning">The earliest reference I could find to the order problem is in a paper by <em>Gary L. Miller</em><sup id="fnref:Mill76" role="doc-noteref"><a href="#fn:Mill76" class="footnote">4</a></sup>, which shows it to be equivalent to factoring (under the Extended Riemann Hypothesis).</p>
\[\Pr
\begin{bmatrix}
\Gho \leftarrow \GenGho(\lambda),\\
g\xleftarrow{\$} \Gho,\\
\ell \leftarrow \Adv(\Gho, g) : \\
g^\ell = \Ghoid
\end{bmatrix} \leq \negl(\lambda)\]
<p><em>Biehl et al.</em> highlight that <em>“The order problem can only be difficult if the order of random elements in G is large with a very high probability.”</em><sup id="fnref:BBHM02" role="doc-noteref"><a href="#fn:BBHM02" class="footnote">5</a></sup></p>
<p><strong>DL assumption implies OA:</strong>
<em>Biehl et al.</em><sup id="fnref:BBHM02:1" role="doc-noteref"><a href="#fn:BBHM02" class="footnote">5</a></sup> also give a reduction from the order problem to the discrete logarithm (DL) problem.
Specifically, given an order problem instance $g$, if one can compute the DL $x$ of $g$ relative to $g^{-1}$ such that $g^x = g^{-1}$, then this gives a solution to the order problem as $g^{x+1} = \Ghoid$.</p>
<p><strong>Strong RSA assumption implies OA:</strong>
This is shown by Boneh et al.<sup id="fnref:BBF18" role="doc-noteref"><a href="#fn:BBF18" class="footnote">6</a></sup> in Lemma 3.
Suppose there is a <strong>generic</strong> adversary $\Adv$ who, given a (random) order problem $g$, solves it by outputting an $\ell$ such that $g^\ell = \Ghoid$.
Then, there exists an adversary $\Badv$ that uses $\Adv$ to solve a random Strong RSA problem $g$.
Specifically, $\Badv$ uses $\Adv$ to break the order problem $g$ and find the $\ell$ mentioned above.
Then, $\Badv$ picks an odd prime $e$ that does not divide $\ell$ and outputs $u = g^{e^{-1} \bmod \ell}$ as the Strong RSA solution.
Note that $e^{-1} \bmod \ell$ is notation for the multiplicative inverse of $e$ modulo $\ell$ (i.e., $e e^{-1} \equiv 1 \pmod \ell$).
This inverse exists since $\gcd(e, \ell)=1$ (because $e$ is prime and does not divide $\ell$).
The Extended Euclidean Algorithm (EEA) can be used to obtain the inverse $a = e^{-1} \bmod \ell$ by finding $(a,b)$ such that:
<!-- TODO: Is \bmod \ell the right notation to use here? Or should we use \pmod \ell? This is what confused me initially. --></p>
\[ae + b\ell = 1 \Rightarrow a = (1 - b\ell)/e\]
<p>Also, note that that $u^e = g$ and so $(u, \ell)$ are a solution for the Strong RSA problem $g$:</p>
\[u^e = (g^{e^{-1} \bmod \ell})^e = (g^\frac{1-b\ell}{e})^e = g^{1-b\ell} = g/(g^{\ell})^b= g\]
<p><strong>RSA assumption implies OA:</strong>
<em>Bieh et al.</em><sup id="fnref:BBHM02:2" role="doc-noteref"><a href="#fn:BBHM02" class="footnote">5</a></sup> give a reduction from the RSA problem to the order problem<sup id="fnref:reduction" role="doc-noteref"><a href="#fn:reduction" class="footnote">7</a></sup>.
Specifically, given an RSA instance $(g, e)$ such that $e$ does not divide $\Ghosz$, if one can compute a multiple $\ell$ of the order of $g$ such that $g^\ell = \Ghoid$, then one can break the RSA instance as follows.
We have two cases.</p>
<ul>
<li><em>First case:</em> If $e$ does not divide $\ell$, then we can break the assumption just like we broke Strong RSA above: output $u = g^{e^{-1} \bmod \ell}$.</li>
<li><em>Second case:</em> If $e$ divides $\ell$, then we can let $\ell’ = \ell / e$. Importantly, since $e$ does not divide the order of $\Gho$, then $\ell’$ is still a multiple of the order of $g$<sup id="fnref:ellprime" role="doc-noteref"><a href="#fn:ellprime" class="footnote">8</a></sup>. Thus, we can output $u = g^{e^{-1} \bmod \ell’}$ as before.</li>
</ul>
<!-- **TODO:** Why do they specifically use class groups in BBHM02, since RSA \QRn subgroups should also work, no?-->
<p><strong>OA implies factoring assumption:</strong>
We have to show that order problem reduces to factoring.
Well, if you have a factoring oracle, you can factor $N=pq$, compute $\phi(N)= (p-1)(q-1)$ and then factor $\phi(N)$ as well.
Next, given an order problem $g$, you know that $g^{\phi(N)} = \Ghoid$.
If only multiples of the order are desired, then the problem is solved.
Otherwise, if the actual order is required, then we know the order has to be a divisor of $\phi(N)$, the order of $\ZNs$.
Thus, one can repeatedly divide out the divisors of $\phi(N)$ from the exponent and check if the result is still $\Ghoid$.
For details, see <a href="https://math.stackexchange.com/questions/1025578/is-there-a-better-way-of-finding-the-order-of-a-number-modulo-n">this post</a> or see Algorithm 4.79 in Chapter 4 of the Handbook of Applied Cryptography<sup id="fnref:MvV96Ch4Pubkey" role="doc-noteref"><a href="#fn:MvV96Ch4Pubkey" class="footnote">9</a></sup>.</p>
<p><strong>Factoring assumption implies OA:</strong>
<em>Miller</em><sup id="fnref:Mill76:1" role="doc-noteref"><a href="#fn:Mill76" class="footnote">4</a></sup> shows that one can factor $N$ if one can compute the order of random group elements in $\ZNs$ (see Theorem 4).
<em>Shor</em><sup id="fnref:Shor97" role="doc-noteref"><a href="#fn:Shor97" class="footnote">10</a></sup> explains this reduction succinctly (see Section 5).
<!--
But the OA assumption asks for multiples of the order, not for the order itself, so the reduction might not have access to the order.
However, the RSA paper[^RSA78] says Miller's result reduction actually works with any multiple of the order.
(However, I'm not sure I fully see this in Miller's paper.)
--></p>
<p><strong>OA in class groups:</strong>
<em>Hamdy and Moller</em><sup id="fnref:HM00" role="doc-noteref"><a href="#fn:HM00" class="footnote">11</a></sup> explain how to solve the order problem in class groups whose class number is smooth.</p>
<!--
Questions and answers:
- Is $\ell$ the actual order or a multiple of the order?
+ Seems like both work, so assuming multiple for generality.
- Is [CF13e] referring to this assumption in their security proof?
+ Yes. They cite [HSS93], but [Mill76] would've been the right reference AFAICT.
- How does the _Hastad et al._[^HSS93] result, which proves most elements in $\ZNs$ are of high order, relate to this assumption?
+ Well, if a significant fraction weren't of high order, then the random $g$ whose order must be found would be easy to solve.
-->
<!--
**TODOs:**
- Strong RSA holds implies OA holds, seems to generalize the rsa.cash statement that Strong RSA implies OA when N is a product of safe primes. Is it true though? Maybe they are saying OA doesn't hold outside of $N$ = product of safe primes, so neither does Strong RSA.
-->
<h3 id="adaptive-root-assumption-ara">Adaptive root assumption (ARA)</h3>
<p>This assumption was introduced by Wesolowski<sup id="fnref:Weso19" role="doc-noteref"><a href="#fn:Weso19" class="footnote">12</a></sup>, who initially called it the <em>“root finding game”</em>.</p>
\[\Pr
\begin{bmatrix}
\Gho \leftarrow \GenGho(\lambda),\\
(g, \mathsf{state}) \xleftarrow{\$} \Adv_0(\Gho),\\
\ell \xleftarrow{\$} \primes{\lambda}, \\
u \leftarrow \Adv_1(\mathsf{state}, g,\ell): \\
g^{1/\ell} = u\ \text{and}\ g\ne \Ghoid
\end{bmatrix} \leq \negl(\lambda)\]
<!--
**TODO:**
- How does the length of the primes affect this assumption? $\lambda$ versus $2\lambda$ versus 3?
-->
<h3 id="the-low-order-assumption-loa">The low-order assumption (LOA)</h3>
<p>This assumption was first used by Pietrzak<sup id="fnref:Piet18e" role="doc-noteref"><a href="#fn:Piet18e" class="footnote">13</a></sup> to construct <em>verifiable delay functions (VDFs)</em> and later formalized by Boneh et al.<sup id="fnref:BBF18A" role="doc-noteref"><a href="#fn:BBF18A" class="footnote">14</a></sup>
Informally, the assumptions says that the <strong>low-order problem</strong> is “hard” to solve.
Specifically, given a group $\Gho$ of hidden-order, it is hard to find $g\in \Gho$ and $\ell \ne 0$ such that $g^{\ell} = \Ghoid$.
<!-- In other words, it is hard to find the order of any element in $\Gho$. --></p>
\[\Pr
\begin{bmatrix}
\Gho \leftarrow \GenGho(\lambda),\\
(g, \ell) \leftarrow \Adv(\Gho) : \\
g^\ell = \Ghoid, g \ne \Ghoid\ \text{and}\ \ell < 2^{\poly(\lambda)}
\end{bmatrix} \leq \negl(\lambda)\]
<p class="warning"><strong>Distinction:</strong> LOA differs from the order assumption (OA) since the LOA adversary gets to choose both $g$ and $\ell$ such that $g^\ell = 1$. In contrast, in OA, the adversary was given a random $g$ and needed to find an $\ell$.</p>
<!--
**TODO:**
- it seems like the RSA paper mentions that any multiple of $\phi(N)$ can be used to factor $N$.
+ Does this immediately imply the low-order assumption must hold in these groups, because if you find $g, x$ such that $g^x = \Ghoid$ for a random group element, then $x$ must be a multiple of $\|\ZNs\| = \phi(N)$, no? Or it could be a multiple of any divisor of $\phi(N) = (p-1)(q-1)$ too I think?
- How come low-order assumption doesn't imply factoring assumption?
-->
<p><strong>Adaptive root assumption implies LOA:</strong>
Boneh et al.<sup id="fnref:BBF18A:1" role="doc-noteref"><a href="#fn:BBF18A" class="footnote">14</a></sup> show that LOA holds in <em>generic groups</em><sup id="fnref:DK02" role="doc-noteref"><a href="#fn:DK02" class="footnote">15</a></sup>, because breaking it implies breaking the adaptive root assumption, which holds generically (see Section 4).</p>
<p><em>Seres and Burcsi</em><sup id="fnref:SB20" role="doc-noteref"><a href="#fn:SB20" class="footnote">16</a></sup> explain that LOA is stronger than factoring arbitrary $N$ because in $\QRn$ all elements are of high order, so factoring $N$ will not help.
However, for particular kinds of $N$, they show that factoring reduces to LOA.</p>
<h2 id="other-notes">Other notes</h2>
<p><em>Hohenberger</em><sup id="fnref:Hoh03" role="doc-noteref"><a href="#fn:Hoh03" class="footnote">17</a></sup> introduces the notion of a <em>pseudo-free group</em> and <em>Rivest</em><sup id="fnref:Rive04" role="doc-noteref"><a href="#fn:Rive04" class="footnote">18</a></sup> shows that if $\ZNs$ is pseudo-free, then $\ZNs$ satisfies Strong RSA and DL.</p>
<p><em>Rabin</em><sup id="fnref:Rabi79" role="doc-noteref"><a href="#fn:Rabi79" class="footnote">19</a></sup> shows that solving polynomial congruences $\phi(x) \equiv 0 \pmod N$ is as hard as factoring $N$ (see Section 5).
He also shows that an algorithm, which given $(y,N)$, computes quadratic residues $x$ such that $x^2 = y \pmod N$ can be used to factor $N$ (see Theorem 1).</p>
<p><em>Shor</em><sup id="fnref:Shor97:1" role="doc-noteref"><a href="#fn:Shor97" class="footnote">10</a></sup> shows that finding the order $\ell$ of a random element $g$ in $\ZNs$ (i.e., $g^\ell = 1$) can be done using a <em>quantum computer</em> in polynomial time.
(Shor specifically requires $\ell$ to be the <em>order</em>: i.e., the smallest such integer.)
Then, Shor uses a reduction by <em>Miller</em><sup id="fnref:Mill76:2" role="doc-noteref"><a href="#fn:Mill76" class="footnote">4</a></sup> to factor $N$, given the quantum oracle for solving the order problem.
Stephanie Blanda gives an <a href="https://blogs.ams.org/mathgradblog/2014/04/30/shors-algorithm-breaking-rsa-encryption/">intuitive explanation</a>.</p>
<p><em>Theorem:</em> For all $a < N$, $\exists r\in \Z$, such that $a^r = 1 \pmod N$ iff. $\gcd(a, N)=1$ (see <a href="https://proofwiki.org/wiki/Integer_has_Multiplicative_Order_Modulo_n_iff_Coprime_to_n">here</a>).
<!-- TODO: Eh, why did I write this down again? --></p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:BW88" role="doc-endnote">
<p><strong>A Key-Exchange System Based on Imaginary Quadratic Fields</strong>, by Johannes Buchmann and Hugh C. Williams, <em>in Journal of Cryptology</em>, 1988 <a href="#fnref:BW88" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Bach84" role="doc-endnote">
<p><strong>Discrete logarithms and factoring</strong>, by Eric Bach, 1984, <a href="https://www2.eecs.berkeley.edu/Pubs/TechRpts/1984/CSD-84-186.pdf">[URL]</a> <a href="#fnref:Bach84" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:RSA78" role="doc-endnote">
<p><strong>A method for obtaining digital signatures and public-key cryptosystems</strong>, by R. L. Rivest and A. Shamir and L. Adleman, <em>in Communications of the {ACM}</em>, 1978, <a href="https://doi.org/10.1145%2F359340.359342">[URL]</a> <a href="#fnref:RSA78" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Mill76" role="doc-endnote">
<p><strong>Riemann’s hypothesis and tests for primality</strong>, by Gary L. Miller, <em>in Journal of Computer and System Sciences</em>, 1976, <a href="http://www.sciencedirect.com/science/article/pii/S0022000076800438">[URL]</a> <a href="#fnref:Mill76" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:Mill76:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a> <a href="#fnref:Mill76:2" class="reversefootnote" role="doc-backlink">↩<sup>3</sup></a></p>
</li>
<li id="fn:BBHM02" role="doc-endnote">
<p><strong>A Signature Scheme Based on the Intractability of Computing Roots</strong>, by Biehl, Ingrid and Buchmann, Johannes and Hamdy, Safuat and Meyer, Andreas, <em>in Designs, Codes and Cryptography</em>, 2002, <a href="https://doi.org/10.1023/A:1014927327846">[URL]</a> <a href="#fnref:BBHM02" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:BBHM02:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a> <a href="#fnref:BBHM02:2" class="reversefootnote" role="doc-backlink">↩<sup>3</sup></a></p>
</li>
<li id="fn:BBF18" role="doc-endnote">
<p><strong>Batching Techniques for Accumulators with Applications to IOPs and Stateless Blockchains</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/1188</em>, 2018, <a href="https://eprint.iacr.org/2018/1188">[URL]</a> <a href="#fnref:BBF18" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:reduction" role="doc-endnote">
<p>A <em>reduction</em> from the RSA problem to the order problem is an algorithm that solves the RSA problem given an oracle for solving the order problem. <a href="#fnref:reduction" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:ellprime" role="doc-endnote">
<p>One way to see why $g^{\ell’} = \Ghoid$ is to note that $g^{\ell} = g^{\ell’ e} = \Ghoid$ and, since $e$ does not divide $\Ghosz$ and $e$ is prime, it follows that $\gcd(e, \Ghosz)$ = 1. Thus, $e$ can be inverted. Raising $g^{\ell’ e} = \Ghoid$ to the inverse $e^{-1}$ gives $g^{\ell’} = \Ghoid$. <a href="#fnref:ellprime" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:MvV96Ch4Pubkey" role="doc-endnote">
<p><strong>Public-Key Parameters</strong>, by Menezes, Alfred J and van Oorschot, Paul C and Vanstone, Scott A, <em>in Handbook of Applied Cryptography</em>, 1996, <a href="http://cacr.uwaterloo.ca/hac/about/chap4.pdf">[URL]</a> <a href="#fnref:MvV96Ch4Pubkey" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Shor97" role="doc-endnote">
<p><strong>Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer</strong>, by Shor, Peter W., <em>in SIAM Journal on Computing</em>, 1997, <a href="https://doi.org/10.1137/S0097539795293172">[URL]</a> <a href="#fnref:Shor97" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:Shor97:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:HM00" role="doc-endnote">
<p><strong>Security of Cryptosystems Based on Class Groups of Imaginary Quadratic Orders</strong>, by Hamdy, Safuat and M"oller, Bodo, <em>in Advances in Cryptology — ASIACRYPT 2000</em>, 2000 <a href="#fnref:HM00" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Weso19" role="doc-endnote">
<p><strong>Efficient Verifiable Delay Functions</strong>, by Wesolowski, Benjamin, <em>in Advances in Cryptology – EUROCRYPT 2019</em>, 2019 <a href="#fnref:Weso19" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Piet18e" role="doc-endnote">
<p><strong>Simple Verifiable Delay Functions</strong>, by Krzysztof Pietrzak, <em>in Cryptology ePrint Archive, Report 2018/627</em>, 2018, <a href="https://eprint.iacr.org/2018/627">[URL]</a> <a href="#fnref:Piet18e" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:BBF18A" role="doc-endnote">
<p><strong>A Survey of Two Verifiable Delay Functions</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/712</em>, 2018, <a href="https://eprint.iacr.org/2018/712">[URL]</a> <a href="#fnref:BBF18A" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:BBF18A:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:DK02" role="doc-endnote">
<p><strong>Generic Lower Bounds for Root Extraction and Signature Schemes in General Groups</strong>, by Damg\aard, Ivan and Koprowski, Maciej, <em>in Advances in Cryptology — EUROCRYPT 2002</em>, 2002 <a href="#fnref:DK02" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:SB20" role="doc-endnote">
<p><strong>A Note on Low Order Assumptions in RSA groups</strong>, by István András Seres and Péter Burcsi, <em>in Cryptology ePrint Archive, Report 2020/402</em>, 2020, <a href="https://eprint.iacr.org/2020/402">[URL]</a> <a href="#fnref:SB20" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Hoh03" role="doc-endnote">
<p><strong>The Cryptographic Impact of Groups with Infeasible Inversion</strong>, by Susan Rae Hohenberger, <em>Master’s Thesis, MIT</em>, 2003 <a href="#fnref:Hoh03" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Rive04" role="doc-endnote">
<p><strong>On the Notion of Pseudo-Free Groups</strong>, by Rivest, Ronald L., <em>in Theory of Cryptography</em>, 2004 <a href="#fnref:Rive04" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Rabi79" role="doc-endnote">
<p><strong>Digitalized Signatures and Public-key Functions as Intractable as Factorization</strong>, by Rabin, M. O., 1979 <a href="#fnref:Rabi79" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Alin TomescuIn this post, we summarize some of the cryptographic hardness assumptions used in hidden-order groups.Kate-Zaverucha-Goldberg (KZG) Constant-Sized Polynomial Commitments2020-05-06T22:38:00+00:002020-05-06T22:38:00+00:00https://alinush.github.io//2020/05/06/kzg-polynomial-commitments<p>Kate, Zaverucha and Goldberg introduced a constant-sized polynomial commitment scheme in 2010<sup id="fnref:KZG10e" role="doc-noteref"><a href="#fn:KZG10e" class="footnote">1</a></sup>.
We refer to this scheme as <strong>KZG</strong> and quickly introduce it below.</p>
<p><strong>Prerequisites:</strong></p>
<ul>
<li>Cyclic groups of prime order and finite fields $\Zp$</li>
<li>Pairings (or bilinear maps)</li>
<li><a href="/2020/03/16/polynomials-for-crypto.html">Polynomials</a></li>
</ul>
<h2 id="trusted-setup">Trusted setup</h2>
<p>To commit to degree $\le \ell$ polynomials, need $\ell$-SDH public parameters:
\((g,g^\tau,g^{\tau^2},\dots,g^{\tau^\ell}) = (g^{\tau^i})_{i\in[0,\ell]}\)</p>
<p>Here, $\tau$ is called the <strong>trapdoor</strong>.
These parameters should be generated via a distributed protocol<sup id="fnref:BCGplus15" role="doc-noteref"><a href="#fn:BCGplus15" class="footnote">2</a></sup>$^,$<sup id="fnref:BGG18" role="doc-noteref"><a href="#fn:BGG18" class="footnote">3</a></sup>$^,$<sup id="fnref:BGM17" role="doc-noteref"><a href="#fn:BGM17" class="footnote">4</a></sup> that outputs just the $g^{\tau^i}$’s and <strong>forgets the trapdoor</strong> $\tau$.</p>
<p>The public parameters are <strong>updatable</strong>: given $g^{\tau^i}$’s, anyone can update them to $g^{\alpha^i}$’s where $\alpha = \tau + \Delta$ by picking a random $\Delta$ and computing:
\(g^{\alpha^i} = \left(g^{\tau^i}\right)^{\Delta^i}\)</p>
<p>This is useful when you want to safely re-use a pre-generated set of public parameters, without trusting that nobody knows the trapdoor.</p>
<h2 id="commitments">Commitments</h2>
<p>Commitment to $\phi(X)=\prod_{i\in[0,d]} \phi_i X^i$ is $c=g^{\phi(\tau)}$ computed as:</p>
\[c=\prod_{i\in[0,\deg{\phi}]} \left(g^{\tau^i}\right)^{\phi_i}\]
<p>Since it is just one group element, the commitment is <em>constant-sized</em>.</p>
<h2 id="evaluation-proofs">Evaluation proofs</h2>
<p>To prove an evaluation $\phi(a) = y$, a <em>quotient polynomial</em> is computed in $O(d)$ time:</p>
\[q(X) = \frac{\phi(X) - y}{X - a}\]
<p>Then, the <em>constant-sized</em> <strong>evaluation proof</strong> is:</p>
\[\pi = g^{q(\tau)}\]
<p>Note that this leverages the <a href="/2020/03/16/polynomials-for-crypto.html#the-polynomial-remainder-theorem">polynomial remainder theorem</a>.</p>
<h3 id="verifying-an-evaluation-proof">Verifying an evaluation proof</h3>
<p>A verifier who has the commitment $c=g^{\phi(\tau)}$ and the proof $\pi=g^{q(\tau)}$ can verify it in <em>constant-time</em> using two pairings:</p>
<p>\begin{align}
e(c / g^y, g) &= e(\pi, g^\tau / g^a) \Leftrightarrow\\<br />
e(g^{\phi(\tau)-y}, g) &= e(g^{q(\tau)}, g^{\tau-a}) \Leftrightarrow\\<br />
e(g,g)^{\phi(\tau)-y} &= e(g,g)^{q(\tau)(\tau-a)}\\<br />
\phi(\tau)-y &= q(\tau)(\tau-a)
\end{align}</p>
<p>This effectively checks that $q(X) = \frac{\phi(X) - y}{X-a}$ by checking this equality holds for $X=\tau$.
In other words, it checks that the <a href="/2020/03/16/polynomials-for-crypto.html#the-polynomial-remainder-theorem">polynomial remainder theorem</a> holds at $X=\tau$.</p>
<h2 id="batch-proofs">Batch proofs</h2>
<p>One can prove multiple evaluations $(\phi(e_i) = y_i)<em>{i\in I}$ for _arbitrary</em> points $e_i$ using a constant-sized <strong>KZG batch proof</strong> $\pi_I = g^{q_I(\tau)}$, where:</p>
<p>\begin{align}
\label{eq:batch-proof-rel}
q_I(X) &=\frac{\phi(X)-R_I(X)}{A_I(X)}\\<br />
A_I(X) &=\prod_{i\in I} (X - e_i)\\<br />
R_I(e_i) &= y_i,\forall i\in I\\<br />
\end{align}</p>
<p>$R_I(X)$ can be interpolated via <a href="/2020/03/16/polynomials-for-crypto.html#lagrange-interpolation">Lagrange interpolation</a> in $O(\vert I\vert\log^2{\vert I\vert})$ time<sup id="fnref:vG13ModernCh10" role="doc-noteref"><a href="#fn:vG13ModernCh10" class="footnote">5</a></sup> as:</p>
<p>\begin{align}
R_I(X)=\sum_{i\in I} y_i \prod_{j\in I,j\ne i}\frac{X - e_j}{e_i - e_j}
\end{align}</p>
<p>$A_I(X)$ can be computed in $O(\vert I \vert \log^2{\vert I \vert})$ time via a <strong>subproduct tree</strong> in $O(\vert I\vert\log^2{\vert I\vert})$ time<sup id="fnref:vG13ModernCh10:1" role="doc-noteref"><a href="#fn:vG13ModernCh10" class="footnote">5</a></sup>, as depicted below (for $\vert I \vert = 8$).
The computation proceeds downwards, in the direction of the arrows, with the $(X-e_i)$ monomials being computed first.</p>
<p><img src="/pictures/accumulator-subproduct-tree-inverted.png" /></p>
<p>Each node in the subproduct tree multiplies the polynomials stored in its two children nodes.
This way, the root polynomial will be exactly $A_I(X)$.
If FFT-based multiplication is used, the time to compute a subproduct tree of size $n$ is:</p>
<p>\begin{align}
T(n) &= 2T(n/2) + O(n\log{n})\\<br />
&= O(n\log^2{n})
\end{align}</p>
<p class="info"><strong>Observation 1:</strong> I believe computing $A_I(X)$ faster for <em>arbitrary</em> points $e_i$ is not possible, but I would be happy to be contradicted!</p>
<p class="info"><strong>Observation 2:</strong> In practice, the algorithms for computing $R_I(X)$ and $A_I(X)$ <em>efficiently</em> would require FFT-based techniques for polynomial division and multiplication, and FFTs are fastest when the finite field $\Zp$ is endowed with $d$th roots of unity for sufficiently high $d$, on the order of the degrees of $R_I(X)$ and $A_I(X)$.</p>
<h3 id="verifying-a-batch-proof">Verifying a batch proof</h3>
<p>The verifier who has the commitment $c$, the evaluations $(e_i, y_i)_{i\in I}$ and a batch proof $\pi_I=g^{q_I(\tau)}$ can verify them as follows.</p>
<ol>
<li>First, he interpolates the <strong>accumulator polynomial</strong> \(A_I(X)=\prod_{i\in I} (X-e_i)\) as discussed above.
Then, commits to in $O(\vert I \vert)$ time:
\begin{align}
a &= g^{A_I(\tau)}
\end{align}</li>
<li>Second, he interpolates $R_I(X)$ s.t. $R_I(e_i)=y_i,\forall i \in I$ as discussed above.
Then, commits to in $O(\vert I \vert)$ time:
\begin{align}
r &= g^{R_I(\tau)}
\end{align}</li>
<li>Third, he checks Equation \ref{eq:batch-proof-rel} holds at $X=\tau$ using two pairings: $e(c / r, g) = e(\pi_I, a)$.</li>
</ol>
<p>Note that:</p>
<p>\begin{align}
e(g^{\phi(\tau)} / g^{R_I(\tau)}, g) &= e(g^{q_I(\tau)}, g^{A_I(\tau)})\Leftrightarrow\\<br />
e(g^{\phi(\tau) - R_I(\tau)}, g) &= e(g,g)^{q_I(\tau) A_I(\tau)}\Leftrightarrow\\<br />
\phi(\tau) - R_I(\tau) &= q_I(\tau) A_I(\tau)
\end{align}</p>
<!-- TODO: ## Commitment and proof homomorphism -->
<h2 id="aggregation-of-proofs">Aggregation of proofs</h2>
<p>For now, we discuss proof aggregation in <a href="/2020/05/06/aggregatable-subvector-commitments-for-stateless-cryptocurrencies.html#aggregating-proofs-into-subvector-profs">a different blog post on building vector commitments (VCs)</a> from KZG.</p>
<!-- TODO: proof updates -->
<!-- TODO: ## Information-theoretic hiding -->
<!-- TODO: ### security proofs -->
<h2 id="applications">Applications</h2>
<p>There are many cryptographic tools one can build using polynomial commitment schemes such as KZG.</p>
<p>Here’s a few we’ve blogged about in the past:</p>
<ul>
<li><a href="/2020/04/02/bilinear-accumulators-for-cryptocurrency.html">Cryptographic accumulators</a></li>
<li>Vector Commitments (VC) schemes <a href="/2020/03/12/towards-scalable-vss-and-dkg.html">with $O(\log{n})$-sized proofs</a> or <a href="/2020/05/06/aggregatable-subvector-commitments-for-stateless-cryptocurrencies.html">with $O(1)$-sized proofs</a></li>
<li><a href="/2020/03/03/range-proofs-from-polynomial-commitments-reexplained.html">Range proofs</a></li>
</ul>
<h4 id="acknowledgements">Acknowledgements</h4>
<p>Many thanks to <a href="http://www.cs.umd.edu/~sshravan/">Shravan Srinivasan</a> for helping improve this post.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:KZG10e" role="doc-endnote">
<p><strong>Polynomial commitments</strong>, by Kate, Aniket and Zaverucha, Gregory M and Goldberg, Ian, 2010, <a href="https://pdfs.semanticscholar.org/31eb/add7a0109a584cfbf94b3afaa3c117c78c91.pdf">[URL]</a> <a href="#fnref:KZG10e" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:BCGplus15" role="doc-endnote">
<p><strong>Secure Sampling of Public Parameters for Succinct Zero Knowledge Proofs</strong>, by E. Ben-Sasson and A. Chiesa and M. Green and E. Tromer and M. Virza, <em>in 2015 IEEE Symposium on Security and Privacy</em>, 2015 <a href="#fnref:BCGplus15" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:BGG18" role="doc-endnote">
<p><strong>A Multi-party Protocol for Constructing the Public Parameters of the Pinocchio zk-SNARK</strong>, by Bowe, Sean and Gabizon, Ariel and Green, Matthew D., <em>in Financial Cryptography and Data Security</em>, 2019 <a href="#fnref:BGG18" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:BGM17" role="doc-endnote">
<p><strong>Scalable Multi-party Computation for zk-SNARK Parameters in the Random Beacon Model</strong>, by Sean Bowe and Ariel Gabizon and Ian Miers, 2017, <a href="https://eprint.iacr.org/2017/1050">[URL]</a> <a href="#fnref:BGM17" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:vG13ModernCh10" role="doc-endnote">
<p><strong>Fast polynomial evaluation and interpolation</strong>, by von zur Gathen, Joachim and Gerhard, Jurgen, <em>in Modern Computer Algebra</em>, 2013 <a href="#fnref:vG13ModernCh10" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:vG13ModernCh10:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
</ol>
</div>Alin TomescuKate, Zaverucha and Goldberg introduced a constant-sized polynomial commitment scheme in 20101. We refer to this scheme as KZG and quickly introduce it below. Prerequisites: Cyclic groups of prime order and finite fields $\Zp$ Pairings (or bilinear maps) Polynomials Trusted setup To commit to degree $\le \ell$ polynomials, need $\ell$-SDH public parameters: \((g,g^\tau,g^{\tau^2},\dots,g^{\tau^\ell}) = (g^{\tau^i})_{i\in[0,\ell]}\) Here, $\tau$ is called the trapdoor. These parameters should be generated via a distributed protocol2$^,$3$^,$4 that outputs just the $g^{\tau^i}$’s and forgets the trapdoor $\tau$. The public parameters are updatable: given $g^{\tau^i}$’s, anyone can update them to $g^{\alpha^i}$’s where $\alpha = \tau + \Delta$ by picking a random $\Delta$ and computing: \(g^{\alpha^i} = \left(g^{\tau^i}\right)^{\Delta^i}\) This is useful when you want to safely re-use a pre-generated set of public parameters, without trusting that nobody knows the trapdoor. Commitments Commitment to $\phi(X)=\prod_{i\in[0,d]} \phi_i X^i$ is $c=g^{\phi(\tau)}$ computed as: \[c=\prod_{i\in[0,\deg{\phi}]} \left(g^{\tau^i}\right)^{\phi_i}\] Since it is just one group element, the commitment is constant-sized. Evaluation proofs To prove an evaluation $\phi(a) = y$, a quotient polynomial is computed in $O(d)$ time: \[q(X) = \frac{\phi(X) - y}{X - a}\] Then, the constant-sized evaluation proof is: \[\pi = g^{q(\tau)}\] Note that this leverages the polynomial remainder theorem. Verifying an evaluation proof A verifier who has the commitment $c=g^{\phi(\tau)}$ and the proof $\pi=g^{q(\tau)}$ can verify it in constant-time using two pairings: \begin{align} e(c / g^y, g) &= e(\pi, g^\tau / g^a) \Leftrightarrow\\ e(g^{\phi(\tau)-y}, g) &= e(g^{q(\tau)}, g^{\tau-a}) \Leftrightarrow\\ e(g,g)^{\phi(\tau)-y} &= e(g,g)^{q(\tau)(\tau-a)}\\ \phi(\tau)-y &= q(\tau)(\tau-a) \end{align} This effectively checks that $q(X) = \frac{\phi(X) - y}{X-a}$ by checking this equality holds for $X=\tau$. In other words, it checks that the polynomial remainder theorem holds at $X=\tau$. Batch proofs One can prove multiple evaluations $(\phi(e_i) = y_i){i\in I}$ for _arbitrary points $e_i$ using a constant-sized KZG batch proof $\pi_I = g^{q_I(\tau)}$, where: \begin{align} \label{eq:batch-proof-rel} q_I(X) &=\frac{\phi(X)-R_I(X)}{A_I(X)}\\ A_I(X) &=\prod_{i\in I} (X - e_i)\\ R_I(e_i) &= y_i,\forall i\in I\\ \end{align} $R_I(X)$ can be interpolated via Lagrange interpolation in $O(\vert I\vert\log^2{\vert I\vert})$ time5 as: \begin{align} R_I(X)=\sum_{i\in I} y_i \prod_{j\in I,j\ne i}\frac{X - e_j}{e_i - e_j} \end{align} $A_I(X)$ can be computed in $O(\vert I \vert \log^2{\vert I \vert})$ time via a subproduct tree in $O(\vert I\vert\log^2{\vert I\vert})$ time5, as depicted below (for $\vert I \vert = 8$). The computation proceeds downwards, in the direction of the arrows, with the $(X-e_i)$ monomials being computed first. Each node in the subproduct tree multiplies the polynomials stored in its two children nodes. This way, the root polynomial will be exactly $A_I(X)$. If FFT-based multiplication is used, the time to compute a subproduct tree of size $n$ is: \begin{align} T(n) &= 2T(n/2) + O(n\log{n})\\ &= O(n\log^2{n}) \end{align} Observation 1: I believe computing $A_I(X)$ faster for arbitrary points $e_i$ is not possible, but I would be happy to be contradicted! Observation 2: In practice, the algorithms for computing $R_I(X)$ and $A_I(X)$ efficiently would require FFT-based techniques for polynomial division and multiplication, and FFTs are fastest when the finite field $\Zp$ is endowed with $d$th roots of unity for sufficiently high $d$, on the order of the degrees of $R_I(X)$ and $A_I(X)$. Verifying a batch proof The verifier who has the commitment $c$, the evaluations $(e_i, y_i)_{i\in I}$ and a batch proof $\pi_I=g^{q_I(\tau)}$ can verify them as follows. First, he interpolates the accumulator polynomial \(A_I(X)=\prod_{i\in I} (X-e_i)\) as discussed above. Then, commits to in $O(\vert I \vert)$ time: \begin{align} a &= g^{A_I(\tau)} \end{align} Second, he interpolates $R_I(X)$ s.t. $R_I(e_i)=y_i,\forall i \in I$ as discussed above. Then, commits to in $O(\vert I \vert)$ time: \begin{align} r &= g^{R_I(\tau)} \end{align} Third, he checks Equation \ref{eq:batch-proof-rel} holds at $X=\tau$ using two pairings: $e(c / r, g) = e(\pi_I, a)$. Note that: \begin{align} e(g^{\phi(\tau)} / g^{R_I(\tau)}, g) &= e(g^{q_I(\tau)}, g^{A_I(\tau)})\Leftrightarrow\\ e(g^{\phi(\tau) - R_I(\tau)}, g) &= e(g,g)^{q_I(\tau) A_I(\tau)}\Leftrightarrow\\ \phi(\tau) - R_I(\tau) &= q_I(\tau) A_I(\tau) \end{align} Aggregation of proofs For now, we discuss proof aggregation in a different blog post on building vector commitments (VCs) from KZG. Applications There are many cryptographic tools one can build using polynomial commitment schemes such as KZG. Here’s a few we’ve blogged about in the past: Cryptographic accumulators Vector Commitments (VC) schemes with $O(\log{n})$-sized proofs or with $O(1)$-sized proofs Range proofs Acknowledgements Many thanks to Shravan Srinivasan for helping improve this post. Polynomial commitments, by Kate, Aniket and Zaverucha, Gregory M and Goldberg, Ian, 2010, [URL] ↩ Secure Sampling of Public Parameters for Succinct Zero Knowledge Proofs, by E. Ben-Sasson and A. Chiesa and M. Green and E. Tromer and M. Virza, in 2015 IEEE Symposium on Security and Privacy, 2015 ↩ A Multi-party Protocol for Constructing the Public Parameters of the Pinocchio zk-SNARK, by Bowe, Sean and Gabizon, Ariel and Green, Matthew D., in Financial Cryptography and Data Security, 2019 ↩ Scalable Multi-party Computation for zk-SNARK Parameters in the Random Beacon Model, by Sean Bowe and Ariel Gabizon and Ian Miers, 2017, [URL] ↩ Fast polynomial evaluation and interpolation, by von zur Gathen, Joachim and Gerhard, Jurgen, in Modern Computer Algebra, 2013 ↩ ↩2Aggregatable Subvector Commitments for Stateless Cryptocurrencies from Lagrange Polynomials2020-05-06T14:00:00+00:002020-05-06T14:00:00+00:00https://alinush.github.io//2020/05/06/aggregatable-subvector-commitments-for-stateless-cryptocurrencies<p class="info"><strong>tl;dr:</strong> We build a vector commitment (VC) scheme from KZG commitments to Lagrange polynomials that has (1) constant-sized, aggregatable proofs, which can all be precomputed in $O(n\log{n})$ time, and (2) linear public parameters, which can be derived from any “powers-of-tau” CRS in $O(n\log{n})$ time.
Importantly, the auxiliary information needed to update proofs (a.k.a. the “update key”) is $O(1)$-sized.
Our scheme is compatible with recent techniques to aggregate subvector proofs across <em>different</em> commitments<sup id="fnref:GRWZ20" role="doc-noteref"><a href="#fn:GRWZ20" class="footnote">1</a></sup>.</p>
<!--more-->
<p>This is joint work with <a href="https://twitter.com/ittaia">Ittai Abraham</a>, <a href="https://twitter.com/VitalikButerin">Vitalik Buterin</a>, <a href="https://twitter.com/drakefjustin">Justin Drake</a>, <a href="https://twitter.com/dankrad">Dankrad Feist</a> and <a href="https://twitter.com/khovr">Dmitry Khovratovich</a>.
Our <strong>full paper</strong> is available online <a href="https://eprint.iacr.org/2020/527">here</a>, has been recently accepted to <a href="https://scn.unisa.it/">SCN’20</a>, and has been presented <a href="https://www.youtube.com/watch?v=Yzs6DEVFTLM">here</a> (25 minutes) and <a href="https://www.youtube.com/watch?v=KGRnpjPjduI&list=PLj80z0cJm8QHm_9BdZ1BqcGbgE-BEn-3Y&index=22&t=0s">here</a> (1 hour).
You can find the slides in <a href="https://github.com/alinush/asvc-talk">this GitHub repo</a>.</p>
<p><strong>A little backstory:</strong>
I’ve been interested in vector commitments (VCs) ever since <a href="https://madars.org/">Madars Virza</a> first showed me how KZG and roots of unity gives rise to a simple VC scheme.
In 2018, I was trying to figure out if VC proofs can be updated fast in such a construction.
I came up with a KZG-based scheme that could update a proof for $v_i$ given a change to any $v_j $.
Unfortunately, it required an $O(n)$-sized, <em>static</em>, <em>update key</em> to do the update.
Since each player $i$ in a stateless cryptocurrency has to update their proof for $v_i$, this $O(n)$-sized update key is an annoying storage burden for that user.</p>
<p>Then, I saw <a href="https://ethresear.ch/t/using-polynomial-commitments-to-replace-state-roots/7095">Vitalik Buterin’s post</a> on using <em>partial fraction decomposition</em> to aggregate KZG proofs.
This was great, since it immediately implied VC proofs can be aggregated.
However, after conversations with <a href="https://twitter.com/ittaia">Ittai Abraham</a> and the Ethereum Research team, it became clear this can also be used to reduce the update key size.
The key ingredient was turning two commitments to $A(X)/(X-i)$ and $A(X)/(X-j)$ into a commitment to $A(X)/\left((X-i)(X-j)\right)$ (see <a href="#updating-proofs">here</a>).
This post explains this technique and how to make it work by taking care of all details (e.g., making update keys verifiable, computing them from the KZG public params efficiently, etc.).</p>
<p hidden="">$$
\newcommand{\vect}[1]{\boldsymbol{\mathrm{#1}}}
\newcommand{\prk}{\mathsf{prk}}
\newcommand{\vrk}{\mathsf{vrk}}
\newcommand{\upk}{\mathsf{upk}}
$$</p>
<!-- \overset{\mathrm{def}}{=} -->
<h1 id="preliminaries">Preliminaries</h1>
<p>Let $[i,j]=\{i,i+1,i+2,\dots,j-1,j\}$ and $[0, n) = [0,n-1]$.
Let $p$ be a sufficiently large prime that denotes the order of our groups.</p>
<p>In this post, beyond basic group theory for cryptographers<sup id="fnref:KL15" role="doc-noteref"><a href="#fn:KL15" class="footnote">2</a></sup> and basic polynomial arithmetic, I will assume you are familiar with a few concepts:</p>
<ul>
<li><strong>Bilinear maps</strong><sup id="fnref:GPS08" role="doc-noteref"><a href="#fn:GPS08" class="footnote">3</a></sup>. Specifically, $\exists$ a bilinear map $e : \G_1 \times \G_2 \rightarrow \G_T$ such that:
<ul>
<li>$\forall u\in \G_1,v\in \G_2, a\in \Zp, b\in \Zp, e(u^a, v^b) = e(u,v)^{ab}$</li>
<li>$e(g_1,g_2)\ne 1_T$ where $g_1,g_2$ are the generators of $\G_1$ and $\G_2$ respectively and $1_T$ is the identity of $\G_T$</li>
</ul>
</li>
<li><strong>KZG</strong><sup id="fnref:KZG10a" role="doc-noteref"><a href="#fn:KZG10a" class="footnote">4</a></sup> <strong>polynomial commitments</strong> (see <a href="/2020/05/06/kzg-polynomial-commitments.html">here</a>),</li>
<li>The <strong>Fast Fourier Transform (FFT)</strong><sup id="fnref:CLRS09" role="doc-noteref"><a href="#fn:CLRS09" class="footnote">5</a></sup> applied to polynomials. Specifically,
<ul>
<li>Suppose $\Zp$ admits a primitive <em>root of unity</em> $\omega$ of order $n$ (i.e., $n \mid p-1$)</li>
<li>Let \(H=\{1, \omega, \omega^2, \omega^3, \dots, \omega^{n-1}\}\) denote the set of all $n$ $n$th roots of unity</li>
<li>Then, FFT can be used to efficiently evaluate any polynomial $\phi(X)$ at all $X\in H$ in $\Theta(n\log{n})$ time
<ul>
<li>i.e., compute all \(\{\phi(\omega^{i-1})\}_{i\in[n]}\)</li>
</ul>
</li>
</ul>
</li>
</ul>
<h1 id="vcs-from-lagrange-polynomials">VCs from Lagrange polynomials</h1>
<p>We build upon a previous line of work on VCs from Lagrange polynomials<sup id="fnref:CDHK15" role="doc-noteref"><a href="#fn:CDHK15" class="footnote">6</a></sup><sup>,</sup><sup id="fnref:KZG10a:1" role="doc-noteref"><a href="#fn:KZG10a" class="footnote">4</a></sup><sup>,</sup><sup id="fnref:Tomescu20" role="doc-noteref"><a href="#fn:Tomescu20" class="footnote">7</a></sup>.</p>
<p>Recall that given a vector $\vect{v} = [v_0, v_1, \dots, v_{n-1}]$, we can interpolate a polynomial $\phi(X)$ such that $\phi(i)=v_i$ as follows:
\begin{align}
\phi(X)=\sum_{i=0}^{n-1} v_i \cdot \lagr_i(X),\ \text{where}\ \lagr_i(X) = \prod_{\substack{j\in [0,n)\\j\ne i}}\frac{X-j}{i-j}
\end{align}</p>
<!-- TODO: add this to polynomial basics -->
<p>It is well-known that this Lagrange representation of $\vect{v}$ naturally gives rise to a <strong>vector commitment (VC)</strong> scheme<sup id="fnref:CF13" role="doc-noteref"><a href="#fn:CF13" class="footnote">8</a></sup>.
The key idea is to commit to $\vect{v}$ by committing to $\phi(X)$ using KZG polynomial commitments (see <a href="/2020/05/06/kzg-polynomial-commitments.html">here</a>).
Then, proving $\phi(i) = v_i$ proves that $v_i$ is the $i$th element in the vector.
Next, we describe how this scheme works in more detail and what features it has.</p>
<h2 id="trusted-setup">Trusted setup</h2>
<p>To set up the VC scheme for committing to any vector of size $n$, use an MPC protocol<sup id="fnref:BGM17" role="doc-noteref"><a href="#fn:BGM17" class="footnote">9</a></sup> to generate public parameters $\left(g^{\tau^i}\right)_{i\in [0,n]}$.
<!-- Note: need to commit to A(X) which has roots at all n i's, so need g^{\tau^n} -->
Then, either:</p>
<ol>
<li>Spend $O(n^2)$ time to compute commitments $\ell_i = g^{\lagr_i(\tau)}$ to all $n$ Lagrange polynomials $\lagr_i(X)$.</li>
<li>Or, “shift” the computation of these commitments into the MPC protocol, losing some efficiency.</li>
</ol>
<p class="warning">We will fix this later by storing $v_i$ at $\phi(\omega_n^i)$, which will allow us to compute all $\ell_i$’s in $O(n\log{n})$ time.</p>
<p>Either way, the <strong>proving key</strong> is $\prk=\left(g^{\tau^i},\ell_i\right)_{i\in[0,n)}$ and will be used to commit to a vector and create proofs.
The <strong>verification key</strong> is $\vrk=(g,g^{\tau})$ and will be used to verify proofs.</p>
<h2 id="committing-to-a-vector">Committing to a vector</h2>
<p>The <strong>commitment</strong> to a vector $\vect{v}$ is just a KZG commitment $c=g^{\phi(\tau)}$ to its polynomial $\phi(X)$.
This can be computed very fast, in $O(n)$ time, given the proving key $\prk$:</p>
<p>\begin{align}
c &= \sum_{i=0}^{n-1} \ell_i^{v_i}\\<br />
&= \sum_{i=0}^{n-1} g^{v_i \cdot \lagr_i(\tau)}\\<br />
&= g^{\prod_{i=0}^{n-1} v_i \cdot \lagr_i(\tau)}\\<br />
&= g^{\phi(\tau)}
\end{align}</p>
<h3 id="updating-the-commitment">Updating the commitment</h3>
<p>KZG commitments and thus vector commitments are <em>homomorphic</em>: given commitments $c$ and $c’$ to $\vect{v}$ and $\vect{v’}$, we can get a commitment $C=c \cdot c’$ to $\vect{v} + \vect{v’}$.</p>
<p>A consequence of this is that we can easily update a commitment $c$ to $c’$, given a change $\delta$ to $v_i$ as:
\begin{align}
c’ = c \cdot \ell_i^{\delta}
\end{align}</p>
<h2 id="constant-sized-proofs">Constant-sized proofs</h2>
<p>To prove that $v_i$ is the $i$th element in $\vect{v}$, we have to prove that $\phi(i)=v_i$.
For this, we need to:</p>
<ol>
<li>Interpolate $\phi(X)$ in $O(n\log^2{n})$ field operations and get its coefficients.</li>
<li>Divide $\phi(X)$ by $X-i$ in $O(n)$ field operations and get a quotient $q_i(X)$ such that $\phi(X)=q_i(X)(X-i) + v_i$ (see the <a href="2020/03/16/polynomials-for-crypto.html#the-polynomial-remainder-theorem">polynomial remainder theorem</a>).</li>
<li>Compute a KZG commitment $\pi_i=g^{q_i(\tau)}$ to $q_i(X)$ using an $O(n)$ time multi-exponentiation</li>
</ol>
<p>The proof will be:
\begin{align}
\pi_i=g^{q_i(\tau)}=g^\frac{\phi(\tau)-v_i}{\tau-i}
\end{align}</p>
<p class="warning">In <em>Appendix D.7</em> in <a href="https://eprint.iacr.org/2020/527">our paper</a>, we show how to compute $\pi_i$ in $O(n)$ time, <em>without interpolating</em> $\phi(X)$ by carefully crafting our public parameters.</p>
<p>To verify the proof, we can check with a pairing that:
\begin{align}
e(c/g^{v_i}, g)=e(\pi_i, g^{\tau}/g^i)
\end{align}</p>
<p>This is equivalent to checking that the <a href="/2020/03/16/polynomials-for-crypto.html#the-polynomial-remainder-theorem">polynomial remainder theorem</a> holds for $\phi(i)$ at $X=\tau$.</p>
<h2 id="constant-sized-i-subvector-proofs">Constant-sized $I$-subvector proofs</h2>
<p>To prove multiple positions $(v_i)_{i\in I}$, an <strong>$I$-subvector proof</strong> $\pi_I$ can be computed using a <a href="/2020/05/06/kzg-polynomial-commitments.html#batch-proofs">KZG batch proof</a> as:</p>
<p>\begin{align}
\pi_I &= g^{q_I(\tau)}=g^\frac{\phi(\tau)-R_I(\tau)}{A_I(\tau)}
\end{align}</p>
<p>For this, the prover has to interpolate the following polynomials in $O(\vert I\vert \log^2{\vert I\vert})$ time:</p>
<p>\begin{align}
A_I(X) &=\prod_{i\in I} (X - i)\\<br />
R_I(X) &=\sum_{i\in I} v_i \prod_{j\in I,j\ne i}\frac{X - j}{i - j}\ \text{s.t.}\ R_I(i) = v_i,\forall i\in I
\end{align}</p>
<p>Verifying the proof can also be done with two pairings:
\begin{align}
e(c/g^{R_I(\tau)}, g)=e(\pi_I, g^{A_I(\tau)})
\end{align}</p>
<p>Note that the verifier has to spend $O(\vert I\vert \log^2{\vert I\vert})$ time to interpolate and commit to $A_I(X)$ and $R_I(X)$.</p>
<p class="warning">Later on, we show how to aggregate an $I$-subvector proof $\pi_I$ from all individual proofs $\pi_i, i\in I$ in $O(\vert I\vert \log^2{\vert I\vert})$ time.</p>
<h1 id="enhancing-lagrange-based-vcs">Enhancing Lagrange-based VCs</h1>
<p>The VC scheme presented so far has several nice features:</p>
<ul>
<li>$O(1)$-sized commitments</li>
<li>$O(n)$-sized proving key and $O(1)$-sized verification key</li>
<li>$O(1)$-sized proofs and $O(1)$-sized $I$-subvector proofs</li>
</ul>
<p>It also has additional features, which we didn’t explain:</p>
<ul>
<li><em>Homomorphic proofs:</em> Suppose we are given (1) a proof $\pi_i$ for $v_i$ w.r.t. a commitment $c$ for $\vect{v}$ and (2) a proof $\pi_i’$ for $v_i’$ w.r.t. to $c’$ for vector $\vect{v’}$. Then, can obtain a proof $\Lambda_i=\pi_i \cdot \pi_i’$ for $v_i + v_i’$ w.r.t. $C=c\cdot c’$, which is a commitment to $\vect{v}+\vect{v’}$.</li>
<li><em>Hiding:</em> can commit to a vector as $g^{\phi(\tau)} h^{r(\tau)}$ to get a commitment that hides all information about $\vect{v}$.
<ul>
<li>Here, will need extra $h^{\tau^i}$’s.</li>
<li>Also, $r(X)$ is a random, degree $n-1$ polynomial.
<!--
Note: degree higher than $n-1$ doesn't do anything extra, AFAICT: if you give $n$ evaluations of $\phi$, you reveal $\phi(X)$ anyway, so no sense in "protecting" r(X).
\
In other applications, it might make sense for r(X) to have degree higher than \phi(X), if you want to hide \phi's degree (I think).
--></li>
</ul>
</li>
</ul>
<p>Nonetheless, applications such as <em>stateless cryptocurrencies</em><sup id="fnref:CPZ18" role="doc-noteref"><a href="#fn:CPZ18" class="footnote">10</a></sup>, require extra features:</p>
<ol>
<li><strong>Aggregatable proofs:</strong> Blocks can be made smaller by aggregating all users’ proofs in a block into a single subvector proof.</li>
<li><strong>Updatable proofs:</strong> In a stateless cryptocurrency, each user $i$ has a proof of her balance stored at position $i$ in the vector. However, since the vector changes after each transaction in the currency, the user must be able to update her proof so it verifies w.r.t. the updated vector commitment.</li>
<li><strong>Precompute <em>all</em> proofs fast:</strong> Proof serving nodes in stateless cryptocurrencies can operate faster if they periodically precompute all proofs rather than updating all $O(n)$ proof after each new block.</li>
<li><strong>Updatable public parameters:</strong> Since many $g^{\tau^i}$’s are already publicly available from previous trusted setup ceremonies implemented via MPC, it would be nice to use them safely by “refreshing” them with additional trusted randomness.</li>
</ol>
<p><a href="https://eprint.iacr.org/2020/527">Our paper</a> adds all these features by carefully making use of roots of unity<sup id="fnref:vG13ModernCh8" role="doc-noteref"><a href="#fn:vG13ModernCh8" class="footnote">11</a></sup>, Fast Fourier Transforms (FFTs)<sup id="fnref:CLRS09:1" role="doc-noteref"><a href="#fn:CLRS09" class="footnote">5</a></sup> and partial fraction decomposition<sup id="fnref:Buterin20UsingPoly" role="doc-noteref"><a href="#fn:Buterin20UsingPoly" class="footnote">12</a></sup>.</p>
<h2 id="aggregating-proofs-into-subvector-profs">Aggregating proofs into subvector profs</h2>
<p>Drake and Buterin<sup id="fnref:Buterin20UsingPoly:1" role="doc-noteref"><a href="#fn:Buterin20UsingPoly" class="footnote">12</a></sup> observe that partial fraction decomposition can be used to aggregate KZG proofs.</p>
<p>Let’s first take a quick look at how partial fraction decomposition works.</p>
<h3 id="partial-fraction-decomposition">Partial fraction decomposition</h3>
<p>Any <em>accumulator polynomial fraction</em> can be decomposed as:
\begin{align}
\frac{1}{\prod_{i\in I} (X-i)} = \sum_{i\in I} c_i \cdot \frac{1}{X-i}
\end{align}</p>
<p>The key question is “What are the $c_i$’s?”
Surprisingly, the answer is given by a slightly tweaked Lagrange interpolation formula on a set of points $I$ <sup id="fnref:BT04" role="doc-noteref"><a href="#fn:BT04" class="footnote">13</a></sup>:</p>
<p>\begin{align}
\lagr_i(X)=\prod_{j\in I, j\ne i} \frac{X-j}{i - j}=\frac{A_I(X)}{A_I’(i) (X-i)},\ \text{where}\ A_I(X)=\prod_{i\in I} (X-i)
\end{align}</p>
<p>Here, $A_I’(X)$ is the derivative of $A_I(X)$ and has the (non-obvious) property that $A_I’(i)=\prod_{j\in I,j\ne i} (i-j)$.
(Check out <a href="/2020/03/12/scalable-bls-threshold-signatures.html#our-quasilinear-time-bls-threshold-signature-aggregation">this post</a> for some intuition on why this tweaked Lagrange formula works.)</p>
<p>Now, let us interpolate the polynomial $\phi(X)=1$ using this new Lagrange formula from a set of $|I|$ points $(v_i, \phi(v_i)=1)_{i\in I}$.
\begin{align}
\phi(X) &= \sum_{i\in I} v_i \lagr_i(X)\Leftrightarrow\\\
1 &= A_I(X)\sum_{i\in[0,n)} \frac{v_i}{A_I’(i)(X-i)}\Leftrightarrow\\<br />
\frac{1}{A_I(X)} &= \sum_{i\in I} \frac{1}{A_I’(i)(X-i)}\Leftrightarrow\\<br />
\frac{1}{A_I(X)} &= \sum_{i\in I} \frac{1}{A_I’(i)}\cdot\frac{1}{(X-i)}\Rightarrow\\<br />
c_i &= \frac{1}{A_I’(i)}
\end{align}</p>
<p>Thus, to compute all $c_i$’s needed to decompose, we need to evaluate $A’(X)$ at all $i\in I$.
Fortunately, this can be done in $O(\vert I\vert \log^2{\vert I\vert})$ field operations using a polynomial multipoint evaluation<sup id="fnref:vG13ModernCh10" role="doc-noteref"><a href="#fn:vG13ModernCh10" class="footnote">14</a></sup>.</p>
<h3 id="applying-partial-fraction-decomposition-to-vc-proofs">Applying partial fraction decomposition to VC proofs</h3>
<p>Recall that an $I$-subvector proof is just a commitment to the following quotient polynomial:</p>
<p>\begin{align}
q_I(X)
&= \phi(X)\frac{1}{A_I(X)}- R_I(X)\frac{1}{A_I(X)}\\<br />
\end{align}</p>
<p>Next, we replace $\frac{1}{A_I(X)}$ with its partial fraction decomposition $\sum_{i\in I} \frac{1}{A_I’(i)(X-i)}$.</p>
<p>\begin{align}
q_I(X)
&= \phi(X)\sum_{i\in I} \frac{1}{A_I’(i)(X-i)} - \left(A_I(X)\sum_{i\in I} \frac{v_i}{A_I’(i)(X-i)}\right)\cdot \frac{1}{A_I(X)} \\<br />
&= \sum_{i\in I} \frac{\phi(X)}{A_I’(i)(X-i)} - \sum_{i\in I} \frac{v_i}{A_I’(i)(X-i)}\\<br />
&= \sum_{i\in I} \frac{1}{A_I’(i)}\cdot \frac{\phi(X) - v_i}{X-i}\\<br />
&= \sum_{i\in I} \frac{1}{A_I’(i)}\cdot q_i(X)
\end{align}</p>
<p>So in the end, we were able to express $q_I(X)$ as a linear combination of $q_i(X)$’s, which are exactly the quotients committed to in the proofs of the $v_i$’s (see <a href="#constant-sized-proofs">here</a>).</p>
<p>Thus, given a set of proofs $(\pi_i)_{i\in I}$ for a bunch of $v_i$’s, we can aggregate them into an $I$-subvector proof $\pi_I$ as:
\begin{align}
\pi_I &= \prod_{i\in I} \pi_i^{\frac{1}{A_I’(i)}}
\end{align}</p>
<p>This takes $O(\vert I\vert \log^2{\vert I\vert})$ field operations to compute all the $c_i$’s, as explained in the previous subsection.</p>
<h2 id="updating-proofs">Updating proofs</h2>
<p>First, recall that a proof $\pi_i$ for $v_i$ is a KZG commitment to:
\begin{align}
q_i(X)=\frac{\phi(X)-v_i}{X-i}
\end{align}</p>
<p>Suppose that $v_j$ changes to $v_j+\delta$, thus changing the vector commitment and invalidating any proof $\pi_i$.
Thus, we want to be able to update any proof $\pi_i$ to a new proof $\pi_i’$ that verifies w.r.t. the updated commitment.
Note that we must consider two cases:</p>
<ol>
<li>$i=j$</li>
<li>$i\ne j$.</li>
</ol>
<p>We refer to the party updating their proof $\pi_i$ as the <strong>proof updater</strong>.</p>
<h3 id="the-ij-case">The $i=j$ case</h3>
<p>Let’s see how the quotient polynomial $q_i’(X)$ in the updated proof $\pi_i’$ relates to the original quotient $q_i(X)$:
\begin{align}
q_i’(X)
&=\frac{\phi’(X)-(v_i+\delta)}{X-i}\\<br />
& =\frac{\left(\phi(X) + \delta\lagr_i(X)\right) - v_i -\delta}{X-i}\\<br />
&=\frac{\phi(X) - v_i}{X-i}-\frac{\delta(\lagr_i(X)-1)}{X-i}\\<br />
&= q_i(X) + \delta\left(\frac{\lagr_i(X)-1}{X-i}\right)
\end{align}</p>
<p>Observe that if we include KZG commitments $u_i$ to $\frac{\lagr_i(X)-1}{X-i}$ in our public parameters, then we can update $\pi_i$ to $\pi_i’$ as:
\begin{align}
\pi_i’ = \pi_i \cdot \left(u_i\right)^{\delta}
\end{align}</p>
<p>We include a commitment $u_i$ as part of each user $i$’s update key $\upk_i = u_i = g^\frac{\lagr_i(\tau)-1}{\tau-i}$.
This way, each user $i$ can update her proof after a change to their own $v_i$.
This leaves us with handling updates to $v_j$ for $j\ne i$.
We handle this next by including additional information in $\upk_i$.</p>
<h3 id="the-ine-j-case">The $i\ne j$ case</h3>
<p>Again, let’s see how $q_i’(X)$ relates to the original $q_i(X)$, but after a change $\delta$ at position $j\ne i$:
\begin{align}
q_i’(X)
&=\frac{\phi’(X)-v_i}{X-i}\\<br />
&=\frac{\left(\phi(X) + \delta\lagr_j(X)\right) - v_i}{X-i}\\<br />
&=\frac{\phi(X) - v_i}{X-i}-\frac{\delta\lagr_j(X)}{X-i}\\<br />
&= q_i(X) + \delta\left(\frac{\lagr_j(X)}{X-i}\right)
\end{align}</p>
<p>This time we are in a bit of pickle because there are $O(n^2)$ possible polynomials $U_{i,j}(X) = \frac{\lagr_j(X)}{X-i}$
Let, $u_{i,j}=g^{U_{i,j}(\tau)}$ denote their commitments.
This would mean we’d need each user $i$ to have $n-1$ $u_{i,j}$’s: one for each $j\in[0,n),j\ne i$.
Then, for any change $\delta$ to $v_j$, user $i$ could update its $\pi_i$ to $\pi_i’$ as:
\begin{align}
\pi_i’ = \pi_i \cdot \left(u_{i,j}\right)^{\delta}
\end{align}</p>
<p>However, this would mean each user $i$’s update key is $\upk_i = (u_i, (u_{i,j})_{j\in [0,n),j\ne i})$ and is $O(n)$-sized.
This makes it impractical for use in applications such as stateless cryptocurrencies, where each user $i$ has to include their $\upk_i$ in every transaction they issue.</p>
<h4 id="re-constructing-u_ij-fast">Re-constructing $u_{i,j}$ fast</h4>
<p>Fortunately, by putting additional information in $\upk_i$ and $\upk_j$, we can help user $i$ reconstruct $u_{i,j}$ in $O(1)$ time.
Let $A(X)=\prod_{i\in [0,n)} (X-i)$ be the accumulator polynomial over all $i$’s.
Let $A’(X)$ be its derivative and store the evaluation $A’(i)$ in each user’s $\upk_i$.
Additionally, store $a_i = g^\frac{A(\tau)}{\tau-i}$ in each user’s $\upk_i$.
(Note that $a_i$ is just a KZG proof for $A(i) = 0$.)</p>
<p class="error">Computing all $a_i$’s takes $O(n^2)$ time, but we improve this to $O(n\log{n})$ time later using roots of unity.</p>
<p>Next, using the tweaked Lagrange formula from before, rewrite $U_{i,j}(X)$ as:
\begin{align}
U_{i,j}(X)
&=\frac{\lagr_j(X)}{X-i}\\<br />
&= \frac{A(X)}{A’(j)(X-j)(X-i)}\\<br />
&= \frac{1}{A’(j)}\cdot A(X) \cdot \frac{1}{(X-j)(X-i)}
\end{align}</p>
<p>Next, notice that we can decompose $\frac{1}{(X-j)(X-i)}$:
\begin{align}
U_{i,j}(X)
&= \frac{1}{A’(j)}\cdot A(X) \cdot \frac{1}{(X-j)(X-i)}\\<br />
&= \frac{1}{A’(j)}\cdot A(X) \cdot \left(c_j \frac{1}{X-j}+ c_i\frac{1}{X-i}\right)
&= \frac{1}{A’(j)}\cdot \left(c_j \frac{A(X)}{X-j}+ c_i\frac{A(X)}{X-i}\right)
\end{align}</p>
<p>Now, notice that this implies the commitment $u_{i,j}$ can be computed in $O(1)$ time as:
\begin{align}
u_{i,j}
&= \left(a_j^{c_j} \cdot a_i^{c_i}\right)^\frac{1}{A’(j)}
\end{align}</p>
<p>What are $c_i$ and $c_j$? Just define $A_{i,j}(X) = (X-i)(X-j)$, take its derivative $A_{i,j}’(X)=(X-i)+(X-j)$ and, <a href="#partial-fraction-decomposition">as mentioned before</a>, you have $c_i=1/A_{i,j}’(i)=1/(i-j)$ and $c_j=1/A_{i,j}’(j)=1/(j-i)$</p>
<p>Thus, it is sufficient to set each user’s $\upk_i=(u_i, a_i, A’(i))$.</p>
<p class="info">Note that for user $i$ to update their proof, they need not just their own $\upk_i$ but also the $\upk_j$ corresponding to the changed position $j$.
This is fine in settings such as stateless cryptocurrencies, where $\upk_j$ is part of the transaction that sends money from user $i$ to user $j$.</p>
<h2 id="verifiable-update-keys">Verifiable update keys</h2>
<p>In the stateless cryptocurrency setting, it is very important that user $i$ be able verify $\upk_j$ before using it to update her proof.
Similarly, miners should verify the update keys they use for updating the commitment $c$.
(We did not discuss it, but $\upk_i$ can also be used to derive a commitment to $\lagr_i(X)$ needed to update $c$ after a change to $v_i$.)</p>
<p>To verify $\upk_i$, we need to include a commitment $a=g^{A(\tau)}$ to $A(X)$ in the $\vrk$.
This way, each $a_i$ in $\upk_i$ can be verified as a normal KZG proof w.r.t. $a$.
Then, each $u_i$ can also be verified by noticing two things:</p>
<ol>
<li>$u_i$ is just a KZG proof that $\lagr_i(i) = 1$</li>
<li>$a_i$ can be transformed into $\ell_i=g^{\lagr_i(\tau)}$ in $O(1)$ time by exponentiating it with $1/A’(i)$, which is part of $\upk_i$</li>
</ol>
<p>As a result, $u_i$ can now be verified as a KZG proof that $\lagr_i(i) = 1$ against $\ell_i$.</p>
<h2 id="precomputing-all-proofs-fast">Precomputing all proofs fast</h2>
<p>Computing all $n$ constant-sized proofs for $v_i=\phi(i)$ in less than quadratic time seems very difficult.
Fortunately, Feist and Khovratovich<sup id="fnref:FK20" role="doc-noteref"><a href="#fn:FK20" class="footnote">15</a></sup> give a beautiful technique that can do this, subject to the restriction that the evaluation points are roots of unity, rather than $[0,1,\dots, n-1]$.
Thus, if we change our scheme to store $v_i$ at $\phi(\omega^i)$ where $\omega$ is an $n$th primitive root of unity, we can use this technique to compute all VC proofs $(\pi_i)_{i\in [0,n)}$ in $O(n\log{n})$ time.</p>
<p>Furthermore, we can use this same technique to compute all the $a_i$’s from each $\upk_i$ in $O(n\log{n})$ time.</p>
<h2 id="efficiently-computable-and-updatable-public-parameters">Efficiently-computable and updatable public parameters</h2>
<p>Our scheme’s public parameters, consisting of the proving key, verification key and update keys, need to be generated via an MPC protocol<sup id="fnref:BGM17:1" role="doc-noteref"><a href="#fn:BGM17" class="footnote">9</a></sup>, to guarantee nobody learns the trapdoor $\tau$.
Unfortunately, the most efficient MPC protocols only output $g^{\tau^i}$’s.
This means we should (ideally) find a way to derive the remaining public parameters from these $g^{\tau^i}$’s.</p>
<p>First, when using roots of unity, we have $A(X)=\prod_{i\in [0,n)} (X-\omega^i) = X^n - 1$.
Thus, the commitment $a=g^{A(\tau)}$ to $A(X)=X^{n} - 1$ can be computed in $O(1)$ time via an exponentiation.</p>
<p>Second, the commitments $\ell_i=g^{\lagr_i(\tau)}$ to Lagrange polynomials can be computed via a single DFT on the $(g^{\tau^i})$’s.
(See <em>Sec 3.12.3, pg. 97</em> in <a href="https://madars.org/">Madars Virza’s</a> PhD thesis<sup id="fnref:Virza17" role="doc-noteref"><a href="#fn:Virza17" class="footnote">16</a></sup>).
<!-- Also briefly mentioned in BCG+15: Oakland paper I-C-2, page 5 --></p>
<p>Third, each $a_i = g^{A(\tau)/(\tau -\omega^i)}$ is just a bilinear accumulator membership proof for $\omega^i$ w.r.t. $A(X)$.
Thus, all $a_i$’s can be computed in $O(n\log{n})$ time via the Feist-Khovratovich technique<sup id="fnref:FK20:1" role="doc-noteref"><a href="#fn:FK20" class="footnote">15</a></sup>.</p>
<p>Lastly, we need a way to compute all $u_i = g^{\frac{\lagr_i(\tau)-1}{X-\omega^i}}$.
It turns out this is also doable in $O(n\log{n})$ time using an FFT on a carefully-crafted input (see <em>Sec 3.4.5</em> in <a href="https://eprint.iacr.org/2020/527">our paper</a>).</p>
<p>As a last benefit, since our parameters can be derived from $g^{\tau^i}$’s which are <em>updatable</em>, our parameters are updatable.
This is very useful as it allows safe reuse of existing parameters generated for other schemes.</p>
<h1 id="parting-thoughts">Parting thoughts</h1>
<p>Please see <a href="https://eprint.iacr.org/2020/527">our paper</a> for more goodies, including:</p>
<ul>
<li>A formalization of our primitive (in Sec 3.1)</li>
<li>The full algorithms of our VC (in Sec 3.4.4)</li>
<li>A new security definition for KZG batch proofs with a reduction to $n$-SBDH (in Appendix C)</li>
<li>The efficient algorithm for computing the $u_i$’s (in Sec 3.4.5)</li>
<li>A comparison to other VCs (in Table 2)</li>
<li>An survey of existing VC schemes over prime-order groups, with a time complexity analysis (in Appendix D)</li>
<li>A smaller, incomplete survey of existing VC schemes over hidden-order groups (in Appendix D)</li>
</ul>
<h2 id="are-roots-of-unity-necessary">Are roots of unity necessary?</h2>
<p><a href="https://twitter.com/chbpap">Babis Papamanthou</a> asked me a very good question: <em>“What functionality requires the use of roots of unity?”</em>
I hope the last two sections answered that clearly:</p>
<ul>
<li>Can precompute all $n$ VC proofs in quasilinear time</li>
<li>Can derive our public parameters efficiently from the $g^{\tau^i}$’s
<ul>
<li>This includes all $u_i$’s and $a_i$’s needed to update proofs efficiently</li>
</ul>
</li>
<li>Can have (efficiently) updatable public parameters</li>
<li>Can remove $A’(i)$ from $\upk_i$, since $A’(i)=n\omega^{-i}$ (see <em>Appendix A</em> in <a href="https://eprint.iacr.org/2020/527">our paper</a>)</li>
</ul>
<h2 id="future-work">Future work</h2>
<p>It would be very exciting to see by how much this new VC scheme improves the performance of stateless cryptocurrencies such as Edrax<sup id="fnref:CPZ18:1" role="doc-noteref"><a href="#fn:CPZ18" class="footnote">10</a></sup>.</p>
<h1 id="acknowledgements">Acknowledgements</h1>
<p>Special thanks goes to <a href="https://madars.org/">Madars Virza</a> who first introduced me to Lagrange-based VCs in 2017 and helped me with some of the related work.</p>
<h3 id="references">References</h3>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:GRWZ20" role="doc-endnote">
<p><strong>Pointproofs: Aggregating Proofs for Multiple Vector Commitments</strong>, by Sergey Gorbunov and Leonid Reyzin and Hoeteck Wee and Zhenfei Zhang, <em>in Cryptology ePrint Archive, Report 2020/419</em>, 2020, <a href="https://eprint.iacr.org/2020/419">[URL]</a> <a href="#fnref:GRWZ20" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:KL15" role="doc-endnote">
<p><strong>Introduction to Modern Cryptography</strong>, by Jonathan Katz and Yehuda Lindell, 2007 <a href="#fnref:KL15" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:GPS08" role="doc-endnote">
<p><strong>Pairings for cryptographers</strong>, by Steven D. Galbraith and Kenneth G. Paterson and Nigel P. Smart, <em>in Discrete Applied Mathematics</em>, 2008 <a href="#fnref:GPS08" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:KZG10a" role="doc-endnote">
<p><strong>Constant-Size Commitments to Polynomials and Their Applications</strong>, by Kate, Aniket and Zaverucha, Gregory M. and Goldberg, Ian, <em>in ASIACRYPT ‘10</em>, 2010 <a href="#fnref:KZG10a" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:KZG10a:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:CLRS09" role="doc-endnote">
<p><strong>Introduction to Algorithms, Third Edition</strong>, by Cormen, Thomas H. and Leiserson, Charles E. and Rivest, Ronald L. and Stein, Clifford, 2009 <a href="#fnref:CLRS09" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:CLRS09:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:CDHK15" role="doc-endnote">
<p><strong>Composable and Modular Anonymous Credentials: Definitions and Practical Constructions</strong>, by Camenisch, Jan and Dubovitskaya, Maria and Haralambiev, Kristiyan and Kohlweiss, Markulf, <em>in Advances in Cryptology – ASIACRYPT 2015</em>, 2015 <a href="#fnref:CDHK15" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Tomescu20" role="doc-endnote">
<p><strong>How to Keep a Secret and Share a Public Key (Using Polynomial Commitments)</strong>, by Tomescu, Alin, 2020 <a href="#fnref:Tomescu20" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:CF13" role="doc-endnote">
<p><strong>Vector Commitments and Their Applications</strong>, by Catalano, Dario and Fiore, Dario, <em>in Public-Key Cryptography – PKC 2013</em>, 2013 <a href="#fnref:CF13" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:BGM17" role="doc-endnote">
<p><strong>Scalable Multi-party Computation for zk-SNARK Parameters in the Random Beacon Model</strong>, by Sean Bowe and Ariel Gabizon and Ian Miers, <em>in Cryptology ePrint Archive, Report 2017/1050</em>, 2017, <a href="https://eprint.iacr.org/2017/1050">[URL]</a> <a href="#fnref:BGM17" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:BGM17:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:CPZ18" role="doc-endnote">
<p><strong>Edrax: A Cryptocurrency with Stateless Transaction Validation</strong>, by Alexander Chepurnoy and Charalampos Papamanthou and Yupeng Zhang, <em>in Cryptology ePrint Archive, Report 2018/968</em>, 2018 <a href="#fnref:CPZ18" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:CPZ18:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:vG13ModernCh8" role="doc-endnote">
<p><strong>Fast Multiplication</strong>, by von zur Gathen, Joachim and Gerhard, Jurgen, <em>in Modern Computer Algebra</em>, 2013 <a href="#fnref:vG13ModernCh8" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:Buterin20UsingPoly" role="doc-endnote">
<p><strong>Using polynomial commitments to replace state roots</strong>, by Vitalik Buterin, <em>in \url{https://ethresear.ch/t/using-polynomial-commitments-to-replace-state-roots/7095}</em>, 2020, <a href="https://ethresear.ch/t/using-polynomial-commitments-to-replace-state-roots/7095">[URL]</a> <a href="#fnref:Buterin20UsingPoly" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:Buterin20UsingPoly:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:BT04" role="doc-endnote">
<p><strong>Barycentric Lagrange Interpolation</strong>, by Berrut, J. and Trefethen, L., <em>in SIAM Review</em>, 2004 <a href="#fnref:BT04" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:vG13ModernCh10" role="doc-endnote">
<p><strong>Fast polynomial evaluation and interpolation</strong>, by von zur Gathen, Joachim and Gerhard, Jurgen, <em>in Modern Computer Algebra</em>, 2013 <a href="#fnref:vG13ModernCh10" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:FK20" role="doc-endnote">
<p><strong>Fast amortized Kate proofs</strong>, by Dankrad Feist and Dmitry Khovratovich, 2020, <a href="https://github.com/khovratovich/Kate/blob/master/Kate_amortized.pdf">[pdf]</a> <a href="#fnref:FK20" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:FK20:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:Virza17" role="doc-endnote">
<p><strong>On Deploying Succinct Zero-Knowledge Proofs</strong>, by Virza, Madars, 2017 <a href="#fnref:Virza17" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Alin Tomescutl;dr: We build a vector commitment (VC) scheme from KZG commitments to Lagrange polynomials that has (1) constant-sized, aggregatable proofs, which can all be precomputed in $O(n\log{n})$ time, and (2) linear public parameters, which can be derived from any “powers-of-tau” CRS in $O(n\log{n})$ time. Importantly, the auxiliary information needed to update proofs (a.k.a. the “update key”) is $O(1)$-sized. Our scheme is compatible with recent techniques to aggregate subvector proofs across different commitments1. Pointproofs: Aggregating Proofs for Multiple Vector Commitments, by Sergey Gorbunov and Leonid Reyzin and Hoeteck Wee and Zhenfei Zhang, in Cryptology ePrint Archive, Report 2020/419, 2020, [URL] ↩Bilinear Accumulators for Cryptocurrency Enthusiasts2020-04-02T08:10:00+00:002020-04-02T08:10:00+00:00https://alinush.github.io//2020/04/02/bilinear-accumulators-for-cryptocurrency<p class="info"><strong>tl;dr:</strong> We give on overview of <em>bilinear accumulators</em>, a more communication-efficient alternative to Merkle Hash Trees (MHTs) that comes at an increase in computation.
Put simply, bilinear accumulators are commitments to sets with constant-sized (non)membership proofs.</p>
<!-- more -->
<p>For more details, see this full post on <a href="https://decentralizedthoughts.github.io/2020-04-02-bilinear-accumulators-for-cryptocurrency/">Decentralized Thoughts</a>.</p>Alin Tomescutl;dr: We give on overview of bilinear accumulators, a more communication-efficient alternative to Merkle Hash Trees (MHTs) that comes at an increase in computation. Put simply, bilinear accumulators are commitments to sets with constant-sized (non)membership proofs. For more details, see this full post on Decentralized Thoughts.Multiplying a Toeplitz matrix by a vector2020-03-19T14:00:00+00:002020-03-19T14:00:00+00:00https://alinush.github.io//2020/03/19/multiplying-a-vector-by-a-toeplitz-matrix<p>These are some notes on how to efficiently multiply a <em>Toeplitz matrix</em> by a vector.
I was writing these for myself while implementing the <a href="https://github.com/khovratovich/Kate/blob/master/Kate_amortized.pdf">new amortized KZG proofs</a> by Feist and Khovratovich, but I thought they might be useful for you too.</p>
<!--more-->
<h2 id="preliminaries">Preliminaries</h2>
<p>We use column vector notation for all vectors.
If $[a, b, c]$ is a row vector, then $[a,b,c]^T$ denotes its transpose: i.e., the column vector \(\begin{bmatrix}a \\\\\ b \\\\\ c\end{bmatrix}\).</p>
<h2 id="whats-a-toeplitz-and-a-circulant-matrix">What’s a Toeplitz (and a circulant) matrix?</h2>
<p>A <em>Toeplitz matrix</em> (e.g. of size $4\times 4$) looks like this:</p>
<p>\begin{bmatrix}
a_0 & a_{-1} & a_{-2} & a_{-3}\\<br />
a_1 & a_0 & a_{-1} & a_{-2}\\<br />
a_2 & a_1 & a_0 & a_{-1}\\<br />
a_3 & a_2 & a_1 & a_0
\end{bmatrix}</p>
<p>Note the odd use of negative indices here, since typically we usually use positive numbers to index.
It’s just convenient for notation to use negative indices.</p>
<p>In other words, it’s a square matrix where the entries “repeat diagonally.”
A concrete example would be:</p>
<p>\begin{bmatrix}
7 & 11 & 5 & 6 \\<br />
3 & 7 & 11 & 5 \\<br />
8 & 3 & 7 & 11 \\<br />
1 & 8 & 3 & 7
\end{bmatrix}</p>
<p>A <em>circulant matrix</em> $C$ is a special form of Toeplitz matrix:</p>
<p>\begin{bmatrix}
a_0 & a_3 & a_2 & a_1\\<br />
a_1 & a_0 & a_3 & a_2\\<br />
a_2 & a_1 & a_0 & a_3\\<br />
a_3 & a_2 & a_1 & a_0
\end{bmatrix}</p>
<p>In other words, each row is shifted/rotated to the right by 1 entry.
(Or, alternatively, each column is shifted/rotated down by 1 entry.)</p>
<p>In general, note that any circulant matrix $C_n$ of size $n\times n$ has a <em>vector representation</em>:</p>
\[\vec{a_n}=[a_0, a_1, \dots, a_{n-1}]\]
<p>Also, note that a circulant matrix is a particular type of a Toeplitz matrix where $a_{-i} = a_{n-i}, \forall i \in[n-1]$.</p>
<p>Here are two examples of circulant matrices:</p>
\[C_4=\begin{bmatrix}
7 & 11 & 5 & 6 \\\\\
6 & 7 & 11 & 5 \\\\\
5 & 6 & 7 & 11 \\\\\
11 & 5 & 6 & 7
\end{bmatrix},
C_4'=\begin{bmatrix}
7 & 1 & 8 & 3 \\\\\
3 & 7 & 1 & 8 \\\\\
8 & 3 & 7 & 1 \\\\\
1 & 8 & 3 & 7
\end{bmatrix}\]
<p>Importantly, a circulant matrix is <em>diagonalizable</em> by the DFT matrix (although we won’t explain why).</p>
<p>First, recall an example of a DFT matrix (e.g., of size $4 \times 4$):</p>
\[F_4=\begin{bmatrix}
1 & 1 & 1 & 1 \\\\\
1 & (w)^1 & (w)^2 & (w)^3 \\\\\
1 & (w^2)^1 & (w^2)^2 & (w^2)^3 \\\\\
1 & (w^3)^1 & (w^3)^2 & (w^3)^3
\end{bmatrix}\]
<p>What we’re saying is that a circulant matrix $C_n$ of size $n\times n$ can be written as:</p>
\[C_n = (F_n)^{-1} \mathsf{diag}(F_n \vec{a_n}) F_n\]
<p>Here, \(\vec{a_n} = [a_0, \dots, a_{n-1}]\) is the vector representation of $C_n$ as discussed before (see above).
Also, $\mathsf{diag}(F_n \vec{a})$ is the $n\times n$ diagonal matrix whose diagonal entries are the entries from $F_n\vec{a_n}$ (i.e., the entry at position $(i,i)$ is the $i$th entry in $F_n\vec{a_n}$) and all other entries are 0.</p>
<h2 id="multiplying-a-circulant-matrix-by-a-vector">Multiplying a circulant matrix by a vector</h2>
<p>Let $y=\mathsf{DFT}(\vec{x}) = F_n \vec{x}$ denote the DFT of a vector $\vec{x}$ and let $\vec{x}=\mathsf{DFT}^{-1}(y)=F_n^{-1} \vec{y}$ denote the inverse DFT.</p>
<p>If $C_n$ is circulant with vector representation $\vec{a_n}$, then multiplying it by a size-$n$ vector $\vec{x}$ can be written as:</p>
<p>\begin{align}
C_n \vec{x} &= \left((F_n)^{-1} \mathsf{diag}(F_n\vec{a_n}) F_n\right)\vec{x}\\<br />
&= (F_n)^{-1} (\mathsf{diag}(F_n\vec{a_n}) (F_n \vec{x}))\\<br />
&= \mathsf{DFT}^{-1}(\mathsf{diag}(\mathsf{DFT}(\vec{a_n})) \mathsf{DFT}(\vec{x}))\\<br />
&= \mathsf{DFT}^{-1}(\mathsf{diag}(\vec{v}) \vec{y})\\<br />
&= \mathsf{DFT}^{-1}(\vec{v} \circ \vec{y})\\<br />
&= \mathsf{DFT}^{-1}(\vec{u})
\end{align}</p>
<p>In other words, what we must do is:</p>
<ul>
<li>Compute $\vec{y}$ by doing a DFT on $\vec{x}$ (in $\Theta(n\log{n})$ time)</li>
<li>Compute $\vec{v}$ by doing a DFT on $\vec{a_n}$ (in $\Theta(n\log{n})$ time)</li>
<li>Compute the Hadamard product $\vec{u}=\vec{v} \circ \vec{y}$,
<ul>
<li>(Since that’s what happens when you multiply a diagonal matrix by a vector.)</li>
</ul>
</li>
<li>Do an inverse DFT on $\vec{u}$ (in $\Theta(n\log{n})$ time).</li>
</ul>
<p>Thus, we can compute $C_n \vec{x}$ in $\Theta(n\log{n})$ time.</p>
<h2 id="multiplying-a-toeplitz-matrix-by-a-vector">Multiplying a Toeplitz matrix by a vector</h2>
<p>To multiply a Toeplitz matrix $T_n$ by a vector $\vec{x}$, we’ll embed the matrix in a circulant matrix $C_{2n}$ in such a manner that the first $n$ entries of $C_{2n}\vec{x}$ will equal exactly $T_n\vec{x}$.</p>
<p>We’ll use $T_4$ as an example:</p>
\[T_4 = \begin{bmatrix}
a_0 & a_{-1} & a_{-2} & a_{-3}\\\\\
a_1 & a_0 & a_{-1} & a_{-2}\\\\\
a_2 & a_1 & a_0 & a_{-1}\\\\\
a_3 & a_2 & a_1 & a_0
\end{bmatrix}\]
<p>We want to build a circulant matrix $C_8$ from $T_4$ such that:</p>
\[C_8 \begin{bmatrix} \vec{x} \\\\\ \vec{0} \end{bmatrix} = \begin{bmatrix} T_4 \vec{x} \\\\\ ? \end{bmatrix}\]
<p class="info">Note that we don’t care what we get in the last $n$ entries of the result, which we denote with a question mark.
Also, $\vec{0}$ denotes the vector of $n=4$ zeros.</p>
<p>If we had such a $C_8$, then we could multiply it with \(\begin{bmatrix}\vec{x}\\\\\ \vec{0}\end{bmatrix}\) using the $\Theta(n\log{n})$ multiplication algorithm from the previous section and efficiently compute $T_4\vec{x}$.</p>
<p>We’ll build $C_8$ from $T_4$ and some other, to be determined matrix which we denote using $B_4$.</p>
\[C_8 = \begin{bmatrix}
T_4 & B_4 \\\\\
B_4 & T_4
\end{bmatrix}\]
<p>Note that this gives us what we want:
\(C_8 \begin{bmatrix}\vec{x}\\\\\ \vec{0}\end{bmatrix} =
\begin{bmatrix}
T_4 & B_4 \\\\\
B_4 & T_4
\end{bmatrix}
\begin{bmatrix}\vec{x}\\\\\ \vec{0}\end{bmatrix}=
\begin{bmatrix}T_4\vec{x}\\\\\ B_4\vec{x}\end{bmatrix}\)</p>
<p>In other words, the first $n$ entries of the product are indeed equal to $T_4 \vec{x}$, independent of what we pick for $B_4$.</p>
<p>But for us to efficiently compute the product, we’ll need to pick a $B_4$ that makes $C_8$ circulant.
So let’s look at what $C_8$ looks like with just the two $T_4$’s in it:</p>
\[C_8 = \begin{bmatrix}
a_0 & a_{-1} & a_{-2} & a_{-3} & ? & ? & ? & ? \\\\\
a_1 & a_0 & a_{-1} & a_{-2} & ? & ? & ? & ? \\\\\
a_2 & a_1 & a_0 & a_{-1} & ? & ? & ? & ? \\\\\
a_3 & a_2 & a_1 & a_0 & ? & ? & ? & ? \\\\
? & ? & ? & ? & a_0 & a_{-1} & a_{-2} & a_{-3}\\\\\
? & ? & ? & ? & a_1 & a_0 & a_{-1} & a_{-2}\\\\\
? & ? & ? & ? & a_2 & a_1 & a_0 & a_{-1}\\\\\
? & ? & ? & ? & a_3 & a_2 & a_1 & a_0
\end{bmatrix}\]
<p>We can fill in part of the puzzle to keep $C_8$ circulant:</p>
\[C_8 = \begin{bmatrix}
a_0 & a_{-1} & a_{-2} & a_{-3} & ? & ? & ? & ? \\\\\
a_1 & a_0 & a_{-1} & a_{-2} & \mathbf{a_{-3}} & ? & ? & ? \\\\\
a_2 & a_1 & a_0 & a_{-1} & \mathbf{a_{-2}} & \mathbf{a_{-3}} & ? & ? \\\\\
a_3 & a_2 & a_1 & a_0 & \mathbf{a_{-1}} & \mathbf{a_{-2}} & \mathbf{a_{-3}} & ? \\\\
? & \mathbf{a_3} & \mathbf{a_2} & \mathbf{a_1} & a_0 & a_{-1} & a_{-2} & a_{-3}\\\\\
? & ? & \mathbf{a_3} & \mathbf{a_2} & a_1 & a_0 & a_{-1} & a_{-2}\\\\\
? & ? & ? & \mathbf{a_3} & a_2 & a_1 & a_0 & a_{-1}\\\\\
? & ? & ? & ? & a_3 & a_2 & a_1 & a_0
\end{bmatrix}\]
<p>By now, you can tell that $B_4$ can be set to:</p>
\[B_4 = \begin{bmatrix}
? & \mathbf{a_3} & \mathbf{a_2} & \mathbf{a_1} \\\\\
\mathbf{a_{-3}} & ? &\mathbf{a_3} & \mathbf{a_2} \\\\\
\mathbf{a_{-2}} &\mathbf{a_{-3}} & ? &\mathbf{a_3} \\\\\
\mathbf{a_{-1}} & \mathbf{a_{-2}} & \mathbf{a_{-3}} & ?
\end{bmatrix}\]
<p>Since the only constraint for the diagonal elements is to be the same, we’ll set them to $a_0$.
So, the final $C_8$ will be:</p>
\[C_8 = \begin{bmatrix}
a_0 & a_{-1} & a_{-2} & a_{-3} & \mathbf{a_0} & \mathbf{a_3} & \mathbf{a_2} & \mathbf{a_1} \\\\\
a_1 & a_0 & a_{-1} & a_{-2} & \mathbf{a_{-3}} & \mathbf{a_0} & \mathbf{a_3} & \mathbf{a_2} \\\\\
a_2 & a_1 & a_0 & a_{-1} & \mathbf{a_{-2}} & \mathbf{a_{-3}} & \mathbf{a_0} & \mathbf{a_3} \\\\\
a_3 & a_2 & a_1 & a_0 & \mathbf{a_{-1}} & \mathbf{a_{-2}} & \mathbf{a_{-3}} & \mathbf{a_0}\\\\
\mathbf{a_0} & \mathbf{a_3} & \mathbf{a_2} & \mathbf{a_1} & a_0 & a_{-1} & a_{-2} & a_{-3}\\\\\
\mathbf{a_{-3}} & \mathbf{a_0} & \mathbf{a_3} & \mathbf{a_2} & a_1 & a_0 & a_{-1} & a_{-2}\\\\\
\mathbf{a_{-2}} & \mathbf{a_{-3}} & \mathbf{a_0} & \mathbf{a_3} & a_2 & a_1 & a_0 & a_{-1}\\\\\
\mathbf{a_{-1}} & \mathbf{a_{-2}} & \mathbf{a_{-3}} & \mathbf{a_0} & a_3 & a_2 & a_1 & a_0
\end{bmatrix}\]
<p>The question that remains to be answered is what is the <em>vector representation</em> $\vec{a_8}$ of $C_8$, since that’s what we’ll need to efficiently evaluate $C_8\vec{x}$ and thus $T_4\vec{x}$.</p>
<p>The answer is, as before, the elements in the first columns of $C_8$, which are:</p>
\[\vec{a_8}=[ a_0, a_1, a_2, a_3, a_0, a_{-3}, a_{-2}, a_{-1} ]^T\]
<p>Thus, applying the algorithm for circulant matrices from before, what we must do is:</p>
<ul>
<li>Build $\vec{a_{2n}}$ from the entries \(\{a_{n-1}, a_{n-2}, \dots, a_1, a_0, a_{-1}, \dots, a_{-(n-1)}\}\) of the Toeplitz matrix $T_n$</li>
<li>Compute $\vec{y}$ by doing a DFT on $[\vec{x}, \vec{0}]^T$</li>
<li>Compute $\vec{v}$ by doing a DFT on $\vec{a_{2n}}$ (e.g., on $\vec{a_8}$ from above)</li>
<li>Compute the Hadamard product $\vec{u}=\vec{v} \circ \vec{y}$,</li>
<li>Do an inverse DFT on $\vec{u}$</li>
<li>The product $T_n \vec{x}$ consists of the first $n$ entries of the resulting vector</li>
</ul>Alin TomescuThese are some notes on how to efficiently multiply a Toeplitz matrix by a vector. I was writing these for myself while implementing the new amortized KZG proofs by Feist and Khovratovich, but I thought they might be useful for you too.