Jekyll2022-07-28T16:33:31+00:00https://alinush.github.io//feed.xmlAlin TomescuTBD Alin TomescuLagrange interpolation2022-07-28T10:38:00+00:002022-07-28T10:38:00+00:00https://alinush.github.io//2022/07/28/lagrange-interpolation<p>Recall from <a href="/2020/03/16/polynomials-for-crypto.html">our basics discussion</a> that a <strong>polynomial</strong> $\phi$ of <strong>degree</strong> $d$ is a vector of $d+1$ <strong>coefficients</strong>:</p> <p>\begin{align} \phi &amp;= [\phi_0, \phi_1, \phi_2, \dots, \phi_d] \end{align}</p> <h2 id="how-to-compute-a-polynomials-coefficients-from-a-bunch-of-its-evaluations">How to compute a polynomial’s coefficients from a bunch of its evaluations</h2> <p>Given $n$ pairs $(x_i, y_i)_{i\in[n]}$, one can compute or <em>interpolate</em> a degree $\le n-1$ polynomial $\phi(X)$ such that: $$\phi(x_i)=y_i,\forall i\in[n]$$</p> <p>Specifically, the <em>Lagrange interpolation</em> formula says that: \begin{align} \label{eq:lagrange-formula} \phi(X) &amp;= \sum_{i\in[n]} y_i \cdot \lagr_i(X),\ \text{where}\ \lagr_i(X) = \prod_{j\in[n],j\ne i} \frac{X-x_j}{x_i-x_j} \end{align}</p> <p>This formula is intimidating at first, but there’s a very simple intuition behind it. The key idea is that $\lagr_i(X)$ is defined so that it has two properties:</p> <ol> <li>$\lagr_i(x_i) = 1,\forall i\in[n]$</li> <li>$\lagr_i(x_j) = 0,\forall j \in [n]\setminus\{i\}$</li> </ol> <p>You can actually convince yourself that $\lagr_i(X)$ has these properties by plugging in $x_i$ and $x_j$ to see what happens.</p> <p class="warning"><strong>Important:</strong> The $\lagr_i(X)$ polynomials are dependent on the set of $x_i$’s only (and thus on $n$)! Specifically each $\lagr_i(X)$ has degree $n-1$ and has a root at each $x_j$ when $j\ne i$! In this sense, a better notation for them would be $\lagr_i^{[x_i, n]}(X)$ or $\lagr_i^{[n]}(X)$ to indicate this dependence.</p> <h2 id="example-interpolating-a-polynomial-from-three-evaluations">Example: Interpolating a polynomial from three evaluations</h2> <p>Consider the following example with $n=3$ pairs of points. Then, by the Lagrange formula, we have:</p> $\phi(X) = y_1 \lagr_1(X) + y_2 \lagr_2(X) + y_3 \lagr_3(X)$ <p>Next, by applying the two key properties of $\lagr_i(X)$ from above, you can easily check that $\phi(x_i) = y_i,\forall i\in$: \begin{align} \phi(x_1) &amp;= y_1 \lagr_1(x_1) + y_2 \lagr_2(x_1) + y_3 \lagr_3(x_1) = y_1 \cdot 1 + y_2 \cdot 0 + y_3 \cdot 0 = y_1\\<br /> \phi(x_2) &amp;= y_1 \lagr_1(x_2) + y_2 \lagr_2(x_2) + y_3 \lagr_3(x_2) = y_1 \cdot 0 + y_2 \cdot 1 + y_3 \cdot 0 = y_2\\<br /> \phi(x_3) &amp;= y_1 \lagr_1(x_3) + y_2 \lagr_2(x_3) + y_3 \lagr_3(x_3) = y_1 \cdot 0 + y_2 \cdot 0 + y_3 \cdot 1 = y_3 \end{align}</p> <p>An <strong>important detail</strong> is that the degree of the interpolated $\phi(X)$ is $\le n-1$ and not necessarily exactly equal to $n-1$. To see this, consider interpolating the polynomial $\phi(X)$ such that $\phi(i) = i$ for all $i\in [n]$. In other words, $x_i = y_i = i$.</p> <p>The inspired reader might notice that the polynomial $\phi(X) = X$ could satisfy our constraints. But is this what the Lagrange interpolation will return? After all, the interpolated $\phi(X)$ is a sum of degree $n-1$ polynomials $\lagr_i(X)$, so could it have degree 1? Well, it turns out, yes, because things cancel out. To see this, take a simple example, with $n=3$: \begin{align} \phi(X) &amp;=\sum_{i\in } i \cdot \lagr_i(X) = \sum_{i\in } i \cdot \prod_{j\in\setminus{i}} \frac{X - j}{i - j}\\<br /> &amp;= 1\cdot \frac{X-2}{1-2}\frac{X-3}{1-3} + 2\cdot \frac{X-1}{2-1}\frac{X-3}{2-3} + 3\cdot\frac{X-1}{3-1}\frac{X-2}{3-2}\\<br /> &amp;= \frac{X-2}{-1}\frac{X-3}{-2} + 2\cdot \frac{X-1}{1}\frac{X-3}{-1} + 3\cdot \frac{X-1}{2}\frac{X-2}{1}\\<br /> &amp;= \frac{1}{2}(X-2)(X-3) - 2(X-1)(X-3) + \frac{3}{2}(X-1)(X-2)\\<br /> &amp;= \frac{1}{2}[(X-2)(X-3) + 3(X-1)(X-2)] - 2(X-1)(X-3)\\<br /> &amp;= \frac{1}{2}[(X-2)(4X-6)] - 2(X-1)(X-3)\\<br /> &amp;= (X-2)(2X-3) - 2(X-1)(X-3)\\<br /> &amp;= (2X^2 - 4X - 3X + 6) - 2(X^2 - 4X +3)\\<br /> &amp;= (2X^2 - 7X + 6) - 2X^2 + 8X - 6\\<br /> &amp;= X \end{align}</p> <h2 id="computational-overhead-of-lagrange-interpolation">Computational overhead of Lagrange interpolation</h2> <p>If done naively, interpolating $\phi(X)$ using the Lagrange formula in Equation \ref{eq:lagrange-formula} will take $O(n^2)$ time.</p> <p>However, there are known techniques for computing $\phi(X)$ in $O(n\log^2{n})$ time. We described <strong>part of</strong> these techniques in a <a href="/2020/03/12/scalable-bls-threshold-signatures.html#our-quasilinear-time-bls-threshold-signature-aggregation">previous blog post</a>, but for the full techniques please refer to the <em>“Modern Computer Algebra”</em> book<sup id="fnref:vG13ModernCh10" role="doc-noteref"><a href="#fn:vG13ModernCh10" class="footnote" rel="footnote">1</a></sup>.</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:vG13ModernCh10" role="doc-endnote"> <p><strong>Fast polynomial evaluation and interpolation</strong>, by von zur Gathen, Joachim and Gerhard, Jurgen, <em>in Modern Computer Algebra</em>, 2013 <a href="#fnref:vG13ModernCh10" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div>Alin TomescuRecall from our basics discussion that a polynomial $\phi$ of degree $d$ is a vector of $d+1$ coefficients: \begin{align} \phi &amp;= [\phi_0, \phi_1, \phi_2, \dots, \phi_d] \end{align} How to compute a polynomial’s coefficients from a bunch of its evaluations Given $n$ pairs $(x_i, y_i)_{i\in[n]}$, one can compute or interpolate a degree $\le n-1$ polynomial $\phi(X)$ such that: $$\phi(x_i)=y_i,\forall i\in[n]$$ Specifically, the Lagrange interpolation formula says that: \begin{align} \label{eq:lagrange-formula} \phi(X) &amp;= \sum_{i\in[n]} y_i \cdot \lagr_i(X),\ \text{where}\ \lagr_i(X) = \prod_{j\in[n],j\ne i} \frac{X-x_j}{x_i-x_j} \end{align} This formula is intimidating at first, but there’s a very simple intuition behind it. The key idea is that $\lagr_i(X)$ is defined so that it has two properties: $\lagr_i(x_i) = 1,\forall i\in[n]$ $\lagr_i(x_j) = 0,\forall j \in [n]\setminus\{i\}$ You can actually convince yourself that $\lagr_i(X)$ has these properties by plugging in $x_i$ and $x_j$ to see what happens. Important: The $\lagr_i(X)$ polynomials are dependent on the set of $x_i$’s only (and thus on $n$)! Specifically each $\lagr_i(X)$ has degree $n-1$ and has a root at each $x_j$ when $j\ne i$! In this sense, a better notation for them would be $\lagr_i^{[x_i, n]}(X)$ or $\lagr_i^{[n]}(X)$ to indicate this dependence. Example: Interpolating a polynomial from three evaluations Consider the following example with $n=3$ pairs of points. Then, by the Lagrange formula, we have: $\phi(X) = y_1 \lagr_1(X) + y_2 \lagr_2(X) + y_3 \lagr_3(X)$ Next, by applying the two key properties of $\lagr_i(X)$ from above, you can easily check that $\phi(x_i) = y_i,\forall i\in$: \begin{align} \phi(x_1) &amp;= y_1 \lagr_1(x_1) + y_2 \lagr_2(x_1) + y_3 \lagr_3(x_1) = y_1 \cdot 1 + y_2 \cdot 0 + y_3 \cdot 0 = y_1\\ \phi(x_2) &amp;= y_1 \lagr_1(x_2) + y_2 \lagr_2(x_2) + y_3 \lagr_3(x_2) = y_1 \cdot 0 + y_2 \cdot 1 + y_3 \cdot 0 = y_2\\ \phi(x_3) &amp;= y_1 \lagr_1(x_3) + y_2 \lagr_2(x_3) + y_3 \lagr_3(x_3) = y_1 \cdot 0 + y_2 \cdot 0 + y_3 \cdot 1 = y_3 \end{align} An important detail is that the degree of the interpolated $\phi(X)$ is $\le n-1$ and not necessarily exactly equal to $n-1$. To see this, consider interpolating the polynomial $\phi(X)$ such that $\phi(i) = i$ for all $i\in [n]$. In other words, $x_i = y_i = i$. The inspired reader might notice that the polynomial $\phi(X) = X$ could satisfy our constraints. But is this what the Lagrange interpolation will return? After all, the interpolated $\phi(X)$ is a sum of degree $n-1$ polynomials $\lagr_i(X)$, so could it have degree 1? Well, it turns out, yes, because things cancel out. To see this, take a simple example, with $n=3$: \begin{align} \phi(X) &amp;=\sum_{i\in } i \cdot \lagr_i(X) = \sum_{i\in } i \cdot \prod_{j\in\setminus{i}} \frac{X - j}{i - j}\\ &amp;= 1\cdot \frac{X-2}{1-2}\frac{X-3}{1-3} + 2\cdot \frac{X-1}{2-1}\frac{X-3}{2-3} + 3\cdot\frac{X-1}{3-1}\frac{X-2}{3-2}\\ &amp;= \frac{X-2}{-1}\frac{X-3}{-2} + 2\cdot \frac{X-1}{1}\frac{X-3}{-1} + 3\cdot \frac{X-1}{2}\frac{X-2}{1}\\ &amp;= \frac{1}{2}(X-2)(X-3) - 2(X-1)(X-3) + \frac{3}{2}(X-1)(X-2)\\ &amp;= \frac{1}{2}[(X-2)(X-3) + 3(X-1)(X-2)] - 2(X-1)(X-3)\\ &amp;= \frac{1}{2}[(X-2)(4X-6)] - 2(X-1)(X-3)\\ &amp;= (X-2)(2X-3) - 2(X-1)(X-3)\\ &amp;= (2X^2 - 4X - 3X + 6) - 2(X^2 - 4X +3)\\ &amp;= (2X^2 - 7X + 6) - 2X^2 + 8X - 6\\ &amp;= X \end{align} Computational overhead of Lagrange interpolation If done naively, interpolating $\phi(X)$ using the Lagrange formula in Equation \ref{eq:lagrange-formula} will take $O(n^2)$ time. However, there are known techniques for computing $\phi(X)$ in $O(n\log^2{n})$ time. We described part of these techniques in a previous blog post, but for the full techniques please refer to the “Modern Computer Algebra” book1. Fast polynomial evaluation and interpolation, by von zur Gathen, Joachim and Gerhard, Jurgen, in Modern Computer Algebra, 2013 &#8617;Feist-Khovratovich technique for computing KZG proofs fast2021-06-17T00:00:00+00:002021-06-17T00:00:00+00:00https://alinush.github.io//2021/06/17/Feist-Khovratovich-technique-for-computing-KZG-proofs-fast<p class="info"><strong>tl;dr:</strong> Given a polynomial $f(X)$ of degree $m$, can we compute all $n$ <a href="/2020/05/06/kzg-polynomial-commitments.html">KZG</a> proofs for $f(\omega^k), k\in[0,n-1]$ in $O(n\log{n})$ time, where $\omega$ is a primitive $n$th root of unity? Dankrad Feist and Dmitry Khovratovich<sup id="fnref:FK20" role="doc-noteref"><a href="#fn:FK20" class="footnote" rel="footnote">1</a></sup> give a resounding ‘yes!’</p> <!--more--> <p hidden="">$$\def\Adv{\mathcal{A}} \def\Badv{\mathcal{B}} \def\vect#1{\mathbf{#1}} %\definecolor{myBlueColor}{HTML}{268BD2} %\definecolor{myPinkColor}{HTML}{D33682} %\definecolor{myGreenColor}{HTML}{859900} %\def\mygreen#1{\color{myGreenColor}{#1}} %\def\mygreen#1{\color{green}{#1}} %\newcommand{\myblue}{\textcolor{myBlueColor}{#1}} %\newcommand{\mypink}{\textcolor{myPinkColor}{#1}}$$</p> <h2 id="preliminaries">Preliminaries</h2> <p>First, read:</p> <ul> <li><a href="/2020/03/16/polynomials-for-crypto.html">Basics of polynomials</a></li> <li>The <a href="https://en.wikipedia.org/wiki/Discrete_Fourier_transform_(general)">Discrete Fourier Transform (DFT)</a> for $n$th roots of unity</li> <li><a href="/2020/05/06/kzg-polynomial-commitments.html">KZG polynomial commitments</a></li> </ul> <p>Notation:</p> <ul> <li>$g$ generates a “bilinear” group $\Gr$ of prime order $p$, endowed with a <em>bilinear map</em> or <em>pairing</em> $e : \Gr\times\Gr\rightarrow\Gr_T$ <ul> <li>we use multiplicative notation here: i.e., $g^a$ denotes composing $g$ with itself $a$ times</li> </ul> </li> <li>there exists a primitive $n$th root of unity $\omega$, where $n$ is a power of two</li> <li>we have $q$-SDH public parameters $\left(g, g^\tau,g^{\tau^2},\dots,g^{\tau^q}\right)$, for $q \le n$</li> <li>we work with polynomials in $\Zp[X]$ of degree $\le n$</li> </ul> <h2 id="related-work">Related work</h2> <p>Other works that give related techniques to compute proofs fast in KZG-like polynomial commitments are:</p> <ul> <li><strong>Authenticated multipoint evaluation trees (AMTs)</strong> (see paper<sup id="fnref:TCZplus20" role="doc-noteref"><a href="#fn:TCZplus20" class="footnote" rel="footnote">2</a></sup> and <a href="/2020/03/12/towards-scalable-vss-and-dkg.html">blogpost</a>)</li> <li><strong>“How to compute all Pointproofs”</strong> (see paper<sup id="fnref:Tome20How" role="doc-noteref"><a href="#fn:Tome20How" class="footnote" rel="footnote">3</a></sup>), much inspired by Feist and Khovratovich’s technique explained in this blogpost</li> </ul> <h2 id="refresher-computing-n-different-kzg-proofs">Refresher: Computing $n$ different KZG proofs</h2> <p>Let $f(X)$ be a polynomial with coefficients $f_i$:</p> <p>\begin{align*} f(X) &amp;= f_m X^m + f_{m-1} X^{m-1} + \cdots f_1 X + f_0\\<br /> &amp;= \sum_{i\in[0,m]} f_i X^i \end{align*}</p> <p><a href="/2020/05/06/kzg-polynomial-commitments.html">Recall that</a> a KZG <strong>evaluation proof</strong> $\pi_i$ for $f(\omega^i)$ is a KZG commitment to a <strong>quotient polynomial</strong> $Q_i(X) = \frac{f(X) - f(\omega^i)}{X-\omega^i}$: \begin{align*} \pi_i = g^{Q_i(\tau)} = g^{\frac{f(\tau) - f(\omega^i)}{\tau-\omega^i}} \end{align*}</p> <p>Computing such a proof takes $O(m)$ time!</p> <p><strong>But what if we want to compute all</strong> $\pi_i, i\in[0, n)$?</p> <p>If done naively, this would take $O(nm)$ time, which is <strong>too expensive!</strong></p> <p class="warning"><strong>Lower bound?</strong> Is this $O(nm)$ time complexity inherent? After all, to compute $\pi_i$ don’t we first need to compute $Q_i(X)$, which takes $O(m)$ time? As you’ll see below, the answer is <strong>“no!”</strong></p> <p>Fortunately, Feist and Khovratovich observe that the $Q_i$’s are algebraically-related and so are their $\pi_i$ KZG commitments! As a result, they observe that computing all $\pi_i$’s does <strong>not</strong> require computing all $Q_i$’s.</p> <p>Below, we explain how their faster, $O(n\log{n})$-time technique works!</p> <p>An important <strong>caveat</strong> is that their technique relies on the evaluation points being $\omega^0,\dots,\omega^{n-1}$.</p> <h2 id="quotient-polynomials-and-their-coefficients">Quotient polynomials and their coefficients</h2> <p>To understand how the $\pi_i$’s relate to one other, let us look at the coefficients of $Q_i(X)$.</p> <p>We can show that when dividing $f$ (of degree $m$) by $(X-\omega^i)$ we obtain a <em>quotient polynomial</em> with coefficients $t_0, t_1, \dots, t_{m-1}$ such that: \begin{align} \label{eq:div-coeffs-1} t_{m-1} &amp;= f_m\\<br /> %t_{m-2} &amp;= f_{m-1} + \omega^i \cdot f_m\\<br /> %t_{m-3} &amp;= f_{m-2} + \omega^i \cdot t_{m-2}\\<br /> % &amp;= f_{m-2} + \omega^i \cdot (f_{m-1} + \omega^i \cdot f_m)\\<br /> % &amp;= f_{m-2} + \omega^i \cdot f_{m-1} + \omega^{2i} \cdot f_m\\<br /> %t_{m-4} &amp;= f_{m-3} + \omega^i \cdot t_{m-3}\\<br /> % &amp;= f_{m-3} + \omega^i \cdot (f_{m-2} + \omega^i \cdot f_{m-1} + \omega^{2i} \cdot f_m)\\<br /> % &amp;= f_{m-3} + \omega^i \cdot f_{m-2} + \omega^{2i} \cdot f_{m-1} + \omega^{3i} \cdot f_m\\\ % &amp; \vdots\\<br /> \label{eq:div-coeffs-2} t_{j} &amp;= f_{j+1} + \omega^i \cdot t_{j+1}, \forall j \in [0, m-1) % &amp; \vdots\\<br /> % t_0 &amp;= f_1 + \omega^i \cdot f_2 + \omega^{2i} f_3 + \dots + \omega^{m-1} f_m \end{align} Note that the $t_i$’s are a function of $f_m, f_{m-1},\dots, f_1$, but not of $f_0$!</p> <details class="info"> <summary> <b style="color: #f5222d">Proof:</b> One could prove by induction that the coefficients above are correct. However, it's easiest to take an example and convince yourself, as shown below for $m=4$. (Click to expand.) </summary> <div style="margin-left: .3em; border-left: .15em solid #f5222d; padding-left: .5em;"><p> Indeed, the quotient obtained when dividing $f(X) = f_3 X^3 + f_2 X^2 + \dots + f_0$ by $X-\omega^i$ exactly matches Equations \ref{eq:div-coeffs-1} and \ref{eq:div-coeffs-2} above: <!-- WARNING: No support for cline in MathJax, so that's why this is a .png --> <div align="center"> <a href="/pictures/fk-division-example.png"> <img style="width:95%" src="/pictures/fk-division-example.png" /> </a> </div> Specifically, the quotient's coefficients, as expected, are: \begin{align} t_2 &amp;= \color{green}{f_3}\\ t_1 &amp;= f_2 + \omega^i t_2\\ &amp;= \color{blue}{f_2 + \omega^i f_3}\\ t_0 &amp;= f_1 + \omega^i t_1 = f_1 + \omega^i \cdot (f_2 + \omega^i f_3)\\ &amp;= \color{pink}{f_1 + \omega^i f_2 + \omega^{2i} f_3} \end{align} </p></div> </details> <h2 id="the-relationship-between-kzg-proofs">The relationship between KZG proofs</h2> <p>Next, let us expand Equations \ref{eq:div-coeffs-1} and \ref{eq:div-coeffs-2} above and get a better sense of the relationship between KZG quotient polynomials: \begin{align} \color{green}{t_{m-1}} &amp;= \underline{f_m}\\<br /> \color{blue}{t_{m-2}} &amp;= f_{m-1} + \omega^i \cdot \color{green}{t_{m-1}} =\nonumber\\<br /> &amp;= \underline{f_{m-1} + \omega^i \cdot f_m}\\<br /> \color{red}{t_{m-3}} &amp;= f_{m-2} + \omega^i \cdot \color{blue}{t_{m-2}}\nonumber\\<br /> &amp;= f_{m-2} + \omega^i \cdot (f_{m-1} + \omega^i \cdot f_m)\nonumber\\<br /> &amp;= \underline{f_{m-2} + \omega^i \cdot f_{m-1} + \omega^{2i} \cdot f_m}\\<br /> t_{m-4} &amp;= f_{m-3} + \omega^i \cdot \color{red}{t_{m-3}}\nonumber\\<br /> &amp;= f_{m-3} + \omega^i \cdot (f_{m-2} + \omega^i \cdot f_{m-1} + \omega^{2i} \cdot f_m)\nonumber\\<br /> &amp;= \underline{f_{m-3} + \omega^i \cdot f_{m-2} + \omega^{2i} \cdot f_{m-1} + \omega^{3i} \cdot f_m}\\\ &amp;\hspace{.55em}\vdots\nonumber\\<br /> % t_{j} &amp;= f_{j+1} + \omega^i \cdot t_{j+1}, \forall j \in [0, m-1)\\<br /> % &amp; \vdots\\<br /> t_1 &amp;= \underline{f_2 + \omega^i \cdot f_3 + \omega^{2i} \cdot f_4 + \dots + \omega^{(m-2)i} \cdot f_m}\\<br /> t_0 &amp;= \underline{f_1 + \omega^i \cdot f_2 + \omega^{2i} \cdot f_3 + \dots + \omega^{(m-1)i} \cdot f_m} \end{align} As you can see above, the quotient polynomial $Q_i(X) = \sum_{j=0}^{m-1} t_j X^j$ obtained when dividing $f(X)$ by $X-\omega^i$ is: \begin{align} Q_i(X) &amp;= f_m \cdot X^{m-1} + {}\\<br /> &amp;+ \left(f_{m-1} + \omega^i \cdot f_m\right) \cdot X^{m-2} + {}\nonumber\\<br /> &amp;+ \left(f_{m-2} + \omega^i \cdot f_{m-1} + \omega^{2i} \cdot f_m\right) \cdot X^{m-3} + {}\nonumber\\<br /> &amp;+ \left(f_{m-3} + \omega^i \cdot f_{m-2} + \omega^{2i} \cdot f_{m-1} + \omega^{3i} \cdot f_m\right) \cdot X^{m-4} + {}\nonumber\\\ &amp;+ \dots + {}\nonumber\\\ &amp;+ \left(f_2 + \omega^i \cdot f_3 + \omega^{2i} \cdot f_4 + \dots + \omega^{(m-2)i} \cdot f_m\right) \cdot X + {}\nonumber\\<br /> &amp;+ \left(f_1 + \omega^i \cdot f_2 + \omega^{2i} \cdot f_3 + \dots + \omega^{(m-1)i} \cdot f_m\right)\nonumber \end{align} Factoring out the roots of unity, we can rearrange this as follows: \begin{align} \label{eq:HX} Q_i(X) &amp;= \left(f_m X^{m-1} + f_{m-1} X^{m-2} + \dots + f_1\right) (\omega^i)^0 + {}\\<br /> &amp;+ \left(f_m X^{m-2} + f_{m-1} X^{m-3} + \dots + f_2\right) (\omega^i)^1 + {}\nonumber\\<br /> &amp;+ \left(f_m X^{m-3} + f_{m-1} X^{m-4} + \dots + f_3\right) (\omega^i)^2 + {}\nonumber\\<br /> &amp;+ \dots + {}\nonumber\\<br /> &amp;+ \left(f_m X + f_{m-1}\right) (\omega^i)^{m-2} + {}\nonumber\\<br /> &amp;+ \left(f_m \right) (\omega^i)^{m-1}\nonumber \end{align} Baptising the polynomials above as $H_j(X)$, we can rewrite as: \begin{align} Q_i(X) &amp;\bydef H_1(X) (\omega^i)^0 + {}\\<br /> &amp;+ H_2(X) (\omega^i)^1 + {}\nonumber\\<br /> &amp;+ \dots + {}\nonumber\\<br /> &amp;+ H_m(X) (\omega^i)^{m-1}\nonumber\\<br /> \end{align} More succinctly, the quotient polynomial is: \begin{align} \label{eq:Qi-poly} Q_i(X) &amp;= \sum_{k=0}^{m-1} H_{j+1}(X) \cdot (\omega^i)^k \end{align}</p> <p class="error"><strong>Note:</strong> At this point, it is not helpful to write down a closed form formula for $H_j(X)$, but we’ll return to it later.</p> <p>Next, let: \begin{align} \label{eq:hj} h_j = g^{H_j(\tau)},\forall j\in[m] \end{align} …denote a KZG commitment to $H_j(X)$. (We are ignoring for now the actual closed-form formula for the $H_j$’s.)</p> <p>Recall that <!--\begin{align}--> $$\pi_i=g^{Q_i(\tau)}$$ <!--\end{align}--> denotes a KZG proof for $\omega^i$.</p> <p>Therefore, applying Equation \ref{eq:Qi-poly} to $\pi_i$’s expression, we get: \begin{align} \label{eq:pi-dft-like} \pi_i = \prod_{j=0}^{m-1} \left(h_{j+1}\right)^{(\omega^i)^j}, \forall i\in[0,n) \end{align}</p> <p>But a close look at Equation \ref{eq:pi-dft-like} reveals it is actually a <strong>Discrete Fourier Transform (DFT)</strong> on the $h_j$’s! Specifically, we can rewrite it as: \begin{align} \label{eq:pi-dft} [ \pi_0, \pi_1, \dots, \pi_{n-1} ] = \mathsf{DFT}_{\Gr}(h_1, h_2, \dots, h_m, h_{m+1},\dots, h_n) \end{align} Here, the extra $h_{m+1},\dots,h_n$ (if any) are just commitments to the zero polynomials: i.e., they are the identity element in $\Gr$. (Also, $\mathsf{DFT}_{\Gr}$ is a DFT on group elements via exponentiations, rather than on field elements via multiplication.)</p> <p class="info"><strong>Time complexity:</strong> Ignoring the time to compute the $h_j$ commitments, which we have not discussed yet, note that the DFT above would only take $O(n\log{n})$ time!</p> <p>This (almost) summarizes the <strong>Feist-Khovratovich (FK)</strong> technique!</p> <p><strong>The key idea?</strong> KZG quotient polynomial commitments are actually related, if the evaluation points are roots of unity. Specifically, these commitments are the output of a single DFT as per Equation \ref{eq:pi-dft}, which can be computed in quasilinear time!</p> <p>However, <strong>one key challenge remains</strong>, which we address next: computing the $h_j$ commitments.</p> <h2 id="computing-the-h_j--gh_jtau-commitments">Computing the $h_j = g^{H_j(\tau)}$ commitments</h2> <p>To see how the $h_j$’s can be computed fast too, let’s rewrite them from Equation \ref{eq:HX}. \begin{align} H_1(X) &amp;= f_m X^{m-1} + f_{m-1} X^{m-2} + \dots + f_1\\<br /> H_2(X) &amp;= f_m X^{m-2} + f_{m-1} X^{m-3} + \dots + f_2\\<br /> H_3(X) &amp;= f_m X^{m-3} + f_{m-1} X^{m-4} + \dots + f_3\\<br /> &amp;\vdots\\<br /> H_m(X) &amp;= f_m X + f_{m-1}\\<br /> H_{m-1}(X) &amp;= f_m \end{align} <strong>Key observation:</strong> We can express the $H_j(X)$ polynomials as a <a href="/2020/03/19/multiplying-a-vector-by-a-toeplitz-matrix.html">Toeplitz matrix product</a> between a matrix $\mathbf{F}$ (of $f(X)$’s coefficients) and a column vector $V(X)$ (of the indeterminate variable $X$): \begin{align} \begin{bmatrix} H_1(X)\\<br /> H_2(X)\\<br /> H_3(X)\\<br /> \vdots\\<br /> H_m(X)\\<br /> H_{m-1}(X)\\<br /> \end{bmatrix} &amp;= \begin{bmatrix} f_m &amp; f_{m-1} &amp; f_{m-2} &amp; f_{m-3} &amp; \dots &amp; f_2 &amp; f_1\\<br /> 0 &amp; f_m &amp; f_{m-1} &amp; f_{m-2} &amp; \dots &amp; f_3 &amp; f_2\\<br /> 0 &amp; 0 &amp; f_m &amp; f_{m-1} &amp; \dots &amp; f_4 &amp; f_3\\<br /> \vdots &amp; &amp; &amp; \ddots &amp; &amp; &amp; \vdots\\<br /> 0 &amp; 0 &amp; 0 &amp; 0 &amp; \dots &amp; f_m &amp; f_{m-1}\\<br /> 0 &amp; 0 &amp; 0 &amp; 0 &amp; \dots &amp; 0 &amp; f_m \end{bmatrix} \cdot \begin{bmatrix} X^{m-1}\\<br /> X^{m-2}\\<br /> X^{m-3}\\<br /> \vdots\\<br /> X\\<br /> 1 \end{bmatrix} \\<br /> &amp;\bydef \mathbf{F} \cdot V(X) \end{align} Therefore, the commitments $h_j$ to the $H_j(X)$’s can also be expressed as a Toeplitz matrix product, where “multiplication” is replaced with “exponentation” and the column vector $V(X)$ is replaced by $V(\tau)$: \begin{align} \begin{bmatrix} h_1\\<br /> h_2\\<br /> h_3\\<br /> \vdots\\<br /> h_m\\<br /> h_{m-1}\\<br /> \end{bmatrix} &amp;= \begin{bmatrix} f_m &amp; f_{m-1} &amp; f_{m-2} &amp; f_{m-3} &amp; \dots &amp; f_2 &amp; f_1\\<br /> 0 &amp; f_m &amp; f_{m-1} &amp; f_{m-2} &amp; \dots &amp; f_3 &amp; f_2\\<br /> 0 &amp; 0 &amp; f_m &amp; f_{m-1} &amp; \dots &amp; f_4 &amp; f_3\\<br /> \vdots &amp; &amp; &amp; \ddots &amp; &amp; &amp; \vdots\\<br /> 0 &amp; 0 &amp; 0 &amp; 0 &amp; \dots &amp; f_m &amp; f_{m-1}\\<br /> 0 &amp; 0 &amp; 0 &amp; 0 &amp; \dots &amp; 0 &amp; f_m \end{bmatrix} \cdot \begin{bmatrix} \tau^{m-1}\\<br /> \tau^{m-2}\\<br /> \tau^{m-3}\\<br /> \vdots\\<br /> \tau\\<br /> 1 \end{bmatrix} \\<br /> &amp;\bydef \mathbf{F} \cdot V(\tau) \end{align} Fortunately, it is well known that such a matrix product can be computed in $O(m\log{m})$ time (incidentally, also via DFTs). If you are curious, in a <a href="/2020/03/19/multiplying-a-vector-by-a-toeplitz-matrix.html">previous blogpost</a>, as well as in a short paper<sup id="fnref:Tome20How:1" role="doc-noteref"><a href="#fn:Tome20How" class="footnote" rel="footnote">3</a></sup>, we explain in detail how this works.</p> <h2 id="conclusion">Conclusion</h2> <p><strong>We are all done!</strong> To summarize, to compute all proofs $\pi_i$ for $f(\omega^i)$, the <em>Feist-Khovratovich (FK)</em> technique<sup id="fnref:FK20:1" role="doc-noteref"><a href="#fn:FK20" class="footnote" rel="footnote">1</a></sup> proceeds as follows:</p> <ol> <li>Computes all $h_j$’s from Equation \ref{eq:hj} in $O(m\log{m})$ time via a Toeplitz matrix product</li> <li>Computes all $\pi_i$’s in $O(n\log{n})$ time via a DFT on the $h_j$’s, as per Equation \ref{eq:pi-dft}</li> </ol> <p>A few things that we could still talk about, but we are out of time:</p> <ul> <li>Implementing this efficiently (see one attempt <a href="https://github.com/alinush/libpolycrypto/blob/fk/libpolycrypto/bench/BenchFk.cpp">here</a>)</li> <li>Optimizing part of the implementation (see Dankrad’s observation’s in <a href="https://twitter.com/alinush407/status/1360228894851305475">this tweet</a>)</li> <li>Other techniques for computing proofs on multiple polynomials from the FK paper<sup id="fnref:FK20:2" role="doc-noteref"><a href="#fn:FK20" class="footnote" rel="footnote">1</a></sup></li> <li>Decreasing KZG verifier time when opening multiple polynomials $(f_i)_{i\in[t]}$ at the same point $x=z^t$, also via the power of DFTs<sup id="fnref:GW21" role="doc-noteref"><a href="#fn:GW21" class="footnote" rel="footnote">4</a></sup></li> </ul> <!-- General formula for $H_j$'s: \begin{align} H_j(X) \bydef \sum_{k=j}^m f_{k} X^{k-j}, \forall j\in[1,m] \end{align} --> <!-- Sanity check: H_1(X) = f_1 X^{1-1} + f_2 X^{2-1} + f_3 X^{3-1} + \dots + f_m X^{m-1} H_2(X) = f_2 X^{2-2} + f_3 X^{3-2} + f_4 X^{4-2} + \dots + f_m X^{m-2} \vdots H_j(X) = \prod_{k = j}^m f_k X^{k-j} \vdots H_{m-1}(X) = f_{m-1} X^{(m-1) - (m-1)} f_m X^{m-(m-1)} = f_{m-1} + f_m X H_m(X) = f_m X^{m-m} = f_m --> <hr /> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:FK20" role="doc-endnote"> <p><strong>Fast amortized Kate proofs</strong>, by Dankrad Feist and Dmitry Khovratovich, 2020, <a href="https://github.com/khovratovich/Kate">[URL]</a> <a href="#fnref:FK20" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:FK20:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a> <a href="#fnref:FK20:2" class="reversefootnote" role="doc-backlink">&#8617;<sup>3</sup></a></p> </li> <li id="fn:TCZplus20" role="doc-endnote"> <p><strong>Towards Scalable Threshold Cryptosystems</strong>, by Alin Tomescu and Robert Chen and Yiming Zheng and Ittai Abraham and Benny Pinkas and Guy Golan Gueta and Srinivas Devadas, <em>in IEEE S\&amp;P’20</em>, 2020 <a href="#fnref:TCZplus20" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Tome20How" role="doc-endnote"> <p><strong>How to compute all Pointproofs</strong>, by Alin Tomescu, <em>in Cryptology ePrint Archive, Report 2020/1516</em>, 2020, <a href="https://eprint.iacr.org/2020/1516">[URL]</a> <a href="#fnref:Tome20How" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:Tome20How:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a></p> </li> <li id="fn:GW21" role="doc-endnote"> <p><strong>fflonk: a Fast-Fourier inspired verifier efficient version of PlonK</strong>, by Ariel Gabizon and Zachary J. Williamson, <em>in Cryptology ePrint Archive, Report 2021/1167</em>, 2021, <a href="https://ia.cr/2021/1167">[URL]</a> <a href="#fnref:GW21" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div>Alin Tomescutl;dr: Given a polynomial $f(X)$ of degree $m$, can we compute all $n$ KZG proofs for $f(\omega^k), k\in[0,n-1]$ in $O(n\log{n})$ time, where $\omega$ is a primitive $n$th root of unity? Dankrad Feist and Dmitry Khovratovich[^FK20] give a resounding ‘yes!’Basic Group Theory2021-04-15T00:00:00+00:002021-04-15T00:00:00+00:00https://alinush.github.io//2021/04/15/basic-group-theory<!-- TODO: totient; gcd's --> <h2 id="multiplicative-inverses-modulo-m">Multiplicative inverses modulo $m$</h2> <p>The multiplicative group of integers modulo $m$ is defined as: \begin{align} \Z_m^* = \{a\ |\ \gcd(a,m) = 1\} \end{align} But why? This is because Euler’s theorem says that: \begin{align} \gcd(a,m) = 1\Rightarrow a^{\phi(m)} = 1 \end{align} This in turn, implies that every element in $\Z_m^*$ has an inverse, since: \begin{align} a\cdot a^{\phi(m) - 1} &amp;= 1 \end{align} Thus, for a prime $p$, all elements in $\Z_p^* = \{1,2,\dots, p-1\}$ have inverses. Specifically, the inverse of $a \in \Z_p^*$ is $a^{p-2}$.</p>Alin TomescuMultiplicative inverses modulo $m$ The multiplicative group of integers modulo $m$ is defined as: \begin{align} \Z_m^* = \{a\ |\ \gcd(a,m) = 1\} \end{align} But why? This is because Euler’s theorem says that: \begin{align} \gcd(a,m) = 1\Rightarrow a^{\phi(m)} = 1 \end{align} This in turn, implies that every element in $\Z_m^*$ has an inverse, since: \begin{align} a\cdot a^{\phi(m) - 1} &amp;= 1 \end{align} Thus, for a prime $p$, all elements in $\Z_p^* = \{1,2,\dots, p-1\}$ have inverses. Specifically, the inverse of $a \in \Z_p^*$ is $a^{p-2}$.What is a Merkle tree?2021-02-25T00:00:00+00:002021-02-25T00:00:00+00:00https://alinush.github.io//2021/02/25/what-is-a-merkle-tree<p class="info"><strong>tl;dr:</strong> In this post, we demystify Merkle trees for beginners. We give simple, illustrated explanations, we detail applications they are used in and reason about their security formally.</p> <!-- more --> <p>For more details, see this full post on <a href="https://decentralizedthoughts.github.io/2020-12-22-what-is-a-merkle-tree/">Decentralized Thoughts</a>.</p>Alin Tomescutl;dr: In this post, we demystify Merkle trees for beginners. We give simple, illustrated explanations, we detail applications they are used in and reason about their security formally. For more details, see this full post on Decentralized Thoughts.Authenticated Dictionaries with Cross-Incremental Proof (Dis)aggregation2020-11-26T00:00:00+00:002020-11-26T00:00:00+00:00https://alinush.github.io//2020/11/26/Authenticated-Dictionaries-with-Cross-Incremental-Proof-Disaggregation<p class="info"><strong>tl;dr:</strong> We build an authenticated dictionary (AD) from Catalano Fiore vector commitments that has constant-sized, aggregatable proofs and supports a stronger notion of cross-incremental proof disaggregation. Our AD could be used for stateless validation in cryptocurrencies with smart contract execution. In a future post, we will extend this AD with stronger security, non-membership proofs and append-only proofs, which makes it applicable to transparency logging.</p> <p>This is joint work with my brilliant (ex-)labmates from MIT, <a href="https://twitter.com/SuperAluex">Alex (Yu) Xia</a> and <a href="https://github.com/znewman01">Zack Newman</a>.</p> <p><strong>Authenticated dictionaries (ADs)</strong> are an important cryptographic primitive which lies at the core of cryptocurrencies such as <em>Ethereum</em> and of transparency logs such as <em>Certificate Transparency (CT)</em>. Typically, ADs are constructed by Merkleizing a lexicographically-ordered data structure such as a binary search tree, a prefix tree or a skip list. However, our work takes a different, more algebraic direction, building upon the <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html">Catalano-Fiore (CF) vector commitment (VC) scheme</a>. This has the advantage of giving us <em>constant-sized</em> proofs which are updatable and aggregatable, with a novel notion of <strong>cross-incrementality</strong>. Importantly, this combination of feautres is not supported by Merkle trees or any other previous VC scheme.</p> <!--more--> <p>In a nutshell, in this post, we:</p> <ul> <li>Extend CF with a larger index space to accommodate dictionary keys, obtaining an authenticated dictionary,</li> <li>Extend our AD to support updating proofs and digests after removing keys from the dictionary,</li> <li>Introduce a novel notion of <strong>cross-incremental proof (dis)aggregation</strong> w.r.t. different ADs</li> </ul> <p>In a future post, we will explain how we:</p> <ul> <li>Strengthen our AD’s security to handle more adversarial settings such as transparency logs,</li> <li>Add proofs of non-membership,</li> <li>Add append-only proofs.</li> </ul> <p>Our algebraic approach is not novel in itself and we relate to the previous line of work that explores building ADs from non-Merkle techniques in <strong>our full paper</strong><sup id="fnref:TXN20e" role="doc-noteref"><a href="#fn:TXN20e" class="footnote" rel="footnote">1</a></sup>. You can also see a quick comparison in our <a href="https://github.com/alinush/authdict-talk/raw/zkstudyclub/talk.pdf">zkStudyClub slides</a>.</p> <h2 id="preliminaries">Preliminaries</h2> <p hidden="">$$\def\Adv{\mathcal{A}} \def\Badv{\mathcal{B}} \def\GenGho{\mathsf{GenGroup}_?} \def\Ghosz{|\Gho|} \def\Ghoid{1_{\Gho}} \def\primes{\mathsf{Primes}} \def\QRn{\mathsf{QR}_N} \def\multirootexp{\mathsf{MultiRootExp}} \def\rootfactor{\mathsf{RootFactor}} \def\vect#1{\mathbf{#1}}$$</p> <p>We often use the following notation:</p> <ul> <li>$\lambda$ denotes the security parameter of our schemes</li> <li>$[n] = \{1,2,\dots, n\}$</li> <li>We denote a vector using a <strong>bolded</strong> variable $\vect{v} = [v_1, \dots, v_n]$</li> <li>$\Gho$ denotes the hidden-order group our constructions use <ul> <li>e.g., $$\Gho = \ZNs =\{a \mathrel\vert \gcd(a,N) = 1\}$$</li> </ul> </li> <li>Let $D$ be a dictionary over a set of keys $K$ that maps each key $k\in K$ to its value $v = D(k)$</li> <li>We sometimes use $k\in D$ to indicate that key $k$ has some value in the dictionary</li> <li>We sometimes use $(k,v)\in D$ notation to indicate that key $k$ has value $v$ in the dictionary</li> <li>We sometimes use $D’ = D\setminus K$ to refer to the new dictionary $D’$ obtained after removing all keys in $K$ (and their values) from the dictionary $D$</li> </ul> <p>This post assumes knowledge of:</p> <ul> <li>Greatest common divisor (GCD) of two integers $x, y$ denoted by $\gcd(x,y)$</li> <li>The Extended Euclidean Algorithm (EEA) for computing Bezout coefficients $x,y\in \Z$ such that $ax + by = \gcd(a,b)$</li> <li><a href="/2020/11/24/RSA-Accumulators.html">RSA accumulators</a> <ul> <li>An RSA accumulator for a set $$T = \{b_1, \dots, b_n\}$$ of elements where each $b_i$ can be hashed to a <em>prime representative</em> $e_i$ is $$a = g^{\prod_{i \in [n]} e_i}$$.</li> <li>An RSA <em>membership witness</em> for $b_i$ is just $$w_i = a^{1/e_i} = g^{\prod_{j\in[n], j\ne i} e_j}$$.</li> <li>To verify it, just check $w_i^{e_i} = a$.</li> <li>Recall all RSA membership witnesses can be computed using an algorithm by <em>Sander et al.</em><sup id="fnref:SSY01" role="doc-noteref"><a href="#fn:SSY01" class="footnote" rel="footnote">2</a></sup> baptised as $$\rootfactor$$ by <em>Boneh et al.</em><sup id="fnref:BBF18" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">3</a></sup>.</li> <li>Specifically, $$\rootfactor(g, (e_i)_{i\in[n]}) = (w_i)_{i\in[n]} = (a^{1/e_i})_{i\in[n]} = \left((g^{\prod_{j\in[n]} e_j})^{1/e_i}\right)_{i\in[n]}$$</li> </ul> </li> <li><a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html">Catalano-Fiore Vector Commitments</a> <ul> <li>Let $H$ be a collision-resistant hash function that maps a vector position $i$ to an $\ell+1$ bit prime $e_i$ such that $2^\ell &lt; e_i &lt; 2^{\ell+1}$</li> <li>The digest of a vector $\vect{v} = [v_1, \dots, v_n]$ is $d(\vect{v}) = (S, \Lambda)$ where: <ul> <li>$S = g^{\prod_{i\in[n]} e_i}$ (i.e., an RSA accumulator over all vector indices $i$)</li> <li>$\Lambda = \prod_{i\in [n]} (S^{1/e_i})^{v_i}$</li> <li>Note that $\Lambda$ is a multi-exponentiation, where: <ul> <li>The bases are RSA witness $S^{1/e_i}$ for $i$,</li> <li>The exponents are the elements $v_i$!</li> </ul> </li> </ul> </li> <li>A proof $\pi_I = (S_I, \Lambda_I)$ for an $I$-subvector $(v_i)_{i\in I}$ is just the digest of $\vect{v}$ without the positions $i\in I$ in it. <ul> <li>$S_I = S^{1/\prod_{i\in I} e_i}$ (i.e., an RSA accumulator over all indices except the ones in $I$)</li> <li>$\Lambda_I = \prod_{i\in[n]\setminus I} (S_I^{1/e_i})^{v_i}$</li> <li>Again, note that $\Lambda_I$ is a multi-exponentiation, where: <ul> <li>The bases are RSA witnesses $S_I^{1/e_i}$ for each $i\in[n]\setminus I$ (but w.r.t. $S_I$)</li> <li>The exponents are elements $v_i$ for all $i\in[n]\setminus I$</li> </ul> </li> </ul> </li> <li>Digests and proofs are <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html#updating-digest">updatable</a></li> <li>Proofs are <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html#disaggregating-proofs">incrementally <em>(dis)aggregatable</em></a></li> </ul> </li> </ul> <h2 id="authenticated-dictionary-ad-schemes">Authenticated dictionary (AD) schemes</h2> <p>First, forget about <em>authenticated dictionaries</em> and let’s talk about good old plain <em>dictionaries</em>! Dictionaries are a set of <strong>key-value pairs</strong> such that each <strong>key</strong> is mapped to one <strong>value</strong>. (We stick to one value per key here, but one could define dictionaries to have multiple values per key too.) The keys are elements of a <strong>key space</strong> which, for our purposes, is the set of strings of length $2\lambda$ bits.</p> <p class="info"><strong>Example:</strong> Your phone’s contacts list is a dictionary: it maps each contact’s phone number (i.e., the key) to that contact’s name (i.e., the value). Similarly, your French-to-English dictionary maps each French word (i.e., the key) to its English counterpart (i.e., the value).</p> <p>Second, what does it mean to <em>authenticate</em> a dictionary? The idea is to outsource storage of the dictionary to a <strong>prover</strong> while allowing one or more <strong>verifiers</strong> to correctly <strong>look up</strong> the values of keys in the dictionary. For this to work, the verifiers must be able to somehow verify the values of keys claimed by the prover. It should be clear that if the verifiers store nothing, there is nothing they can verify these claims against. Thus, verifiers must store <em>something</em>. Since the goal is to outsource storage of the data structure, verifiers will only store a succinct representation of the dictionary called a <strong>digest</strong>. Importantly, while the dictionary might be very large (e.g., the contacts list of a social butterfly), the digest will actually be constant-sized (e.g., 32 bytes).</p> <p>Third, how do verifiers look up in authenticated dictionaries? Verifiers simply ask the prover for a key’s value! Then, the prover replies with the value together with a <strong>lookup proof</strong> that the verifier checks against the digest!</p> <p class="info"><strong>Example:</strong> Some of you might be familiar with Merkle prefix trees. Consider a “sparse” prefix tree that maps each key to a unique leaf. This is best explained by Laurie and Kasper<sup id="fnref:LK15" role="doc-noteref"><a href="#fn:LK15" class="footnote" rel="footnote">4</a></sup> but, simply put, each key is hashed to a unique path in the tree whose leaf stores that key’s value. The <em>digest</em> is the Merkle root hash of this Merkle prefix tree. A <em>lookup proof</em> is the Merkle sibling path to the key’s value in the prefix tree.</p> <h2 id="our-updatable-ad-for-stateless-validation">Our updatable AD for stateless validation</h2> <p>We start with a <strong>simple observation</strong>: the CF VC scheme can be repurposed into an authenticated dictionary scheme by treating the vector indices as the dictionary’s keys<sup id="fnref:obs1" role="doc-noteref"><a href="#fn:obs1" class="footnote" rel="footnote">5</a></sup>. Recall that CF VCs use a collision-resistant hash function $H$ that maps a vector position $i$ to an $(\ell+1)$-bit prime $e_i$ such that $2^\ell &lt; e_i &lt; 2^{\ell+1}$.</p> <p>We let $e_k = H(k)$ for each key $k$ in the dictionary. Then, the dictionary’s digest is: \begin{align} S &amp;= g^{\prod_{k\in D} e_k}\\<br /> c &amp;= \prod_{(k,v) \in D} (S^{1/e_k})^v \end{align} Note that this is just a CF commitment to a “very sparse” vector, with indices in the key space of the dictionary (The key space is of size $2^{2\lambda}$ since it contains all strings of length $2\lambda$ bits.) In other words, the dictionary’s key is the vector’s index while the key’s value is the vector element at that index. Because of this, all the properties of CF VCs carry over to our authenticated dictionary: constant-sized public parameters, incremental proof (dis)aggregation, proof updates and proof precomputation.</p> <p>Nonetheless, we further enhance this AD by making it more updatable and more (dis)aggregatable. We call the resulting AD an <strong>updatable authenticated dictionary (UAD)</strong>.</p> <p class="info">Note that ADs cannot be obtained in this fashion from any VC scheme. For example, <a href="https://alinush.github.io/2020/05/06/aggregatable-subvector-commitments-for-stateless-cryptocurrencies.html">KZG-based VCs</a> do not support a sparse set of vector indices (but nonetheless other techniques<sup id="fnref:Feis20Multi" role="doc-noteref"><a href="#fn:Feis20Multi" class="footnote" rel="footnote">6</a></sup> can be used there). However, some schemes like Catalano-Fiore<sup id="fnref:CF13e" role="doc-noteref"><a href="#fn:CF13e" class="footnote" rel="footnote">7</a></sup> and Boneh et al’s VC<sup id="fnref:BBF18:1" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">3</a></sup> do support sparse indices. Indeed, Boneh et al.<sup id="fnref:BBF18:2" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">3</a></sup> also build an AD on top of their VC scheme, but it is not as (dis)aggregatable as ours.</p> <h3 id="updating-the-digest-after-removals">Updating the digest after removals</h3> <p>One new feature we add is updating the digest after a key and its value are <em>removed</em> from the dictionary. This is very easy to do thanks to the versatility of CF VCs. First, recall that the proof for $(k,v)$ is just the digest of the dictionary $D$ but without $(k,v)$ in it. Thus, if we remove $(k,v)$ from $D$, the new digest is just the proof for $(k,v)$! If we do multiple removals, we can simply <a href="/2020/11/24/Catalano-Fiore-Vector-Commitments.html#aggregating-proofs">aggregate</a> the proofs of all removed keys, which is just the digest of $D$ without those keys in it. Thus, the new digest after multiple removals is simply this aggregated proof!</p> <h3 id="updating-proofs-after-removals">Updating proofs after removals</h3> <p>We also have to add support for updating proofs after a key (and its value) is <em>removed</em> from the dictionary. Let’s say we want to update an aggregated proof $\pi_K$ for a set of keys $K$ after removing a single key $\hat{k}$ with proof $\pi_{\hat{k}}$. Recall that $\pi_K$ is the digest of $D\setminus K$. Since the updated dictionary will be $$D\setminus \{\hat{k}\}$$, the updated proof $\pi_K’$ must be the digest of $$(D\setminus \{\hat{k}\}) \setminus K$$, which is just $$D\setminus (\{\hat{k}\}\cup K)$$.</p> <p>So we must find a way to go from the digest of $D\setminus K$ and of $$D\setminus\{\hat{k}\}$$ to the digest of $$D\setminus (\{\hat{k}\}\cup K)$$. Well, the digest of $$D\setminus (\{\hat{k}\}\cup K)$$ is nothing but the aggregated proof for $K$ and $\hat{k}$. Thus, the updated proof for $K$ is simply the aggregation of the old proof for $K$ with the proof for the removed $\hat{k}$. Naturally, if multiple keys are being removed, then we just aggregate $\pi_K$ with the proofs for each removed key.</p> <p class="warning">One thing we’ve glanced over was that if $$K = \{\hat{k}\}$$, then this proof update doesn’t really work, since we’d be updating the proof for $\hat{k}$ after removing $\hat{k}$ itself. This doesn’t make sense unless we updated $\hat{k}$’s lookup proof into a non-membership proof, which we have not defined yet, but will do so in a future post. <br /> <br /> We’ve also glanced over having $\hat{k}\in K$. But this is not problematic since, in this case, we have $$D\setminus K = D\setminus (\{\hat{k}\}\cup K)$$, so the updated proof $\pi_K’ =\pi_K$.</p> <h3 id="cross-incremental-proof-aggregation">Cross-incremental proof aggregation</h3> <p>Our paper’s main contribution is <em>cross-incremental proof aggregation</em> for our AD, a technique for <strong>incrementally</strong> aggregating lookup proofs <em>across different dictionaries</em>. Recall that we can already (incrementally) aggregate two proofs, one for a set of keys $K_1$ and another for $K_2$, into a single proof for the set of keys $K_1\cup K_2$. For this to work though, these two proofs must be w.r.t. the same dictionary digest $d$. However, in some applications, we’ll be dealing with proofs $\pi_i$, each for a set of keys $K_i$ but w.r.t. their own digest $d_i$. This raises the question of whether such proofs can also be <em>cross-aggregated</em>? Gorbunov et al.<sup id="fnref:GRWZ20e" role="doc-noteref"><a href="#fn:GRWZ20e" class="footnote" rel="footnote">8</a></sup> answer this question positively for vector commitments and our work extends this to authenticated dictionaries.</p> <p class="info"><strong>Example:</strong> In stateless validation for smart contracts<sup id="fnref:GRWZ20e:1" role="doc-noteref"><a href="#fn:GRWZ20e" class="footnote" rel="footnote">8</a></sup>, the $i$th’s smart contract’s memory is represented as a dictionary with digest $d_i$. When this $i$th contract is invoked, the transaction will need to include the subset of memory locations $K_i$ that were accessed by the execution together with their proof $\pi_i$. When multiple transactions are processed, each proof $\pi_i$ will be w.r.t. a different $d_i$. Importantly, instead of including each $\pi_i$ in the mined block, we would ideally like to <em>cross-aggregate</em> all $\pi_i$’s into a single proof $\pi$.</p> <h4 id="proof-of-knowledge-of-co-prime-roots">Proof-of-knowledge of co-prime roots</h4> <p>The key ingredient behind our incremental cross-aggregation is the <strong>proof-of-knowledge of co-prime roots (PoKCR)</strong> protocol by Boneh et al.<sup id="fnref:BBF18:3" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">3</a></sup> Recall that PoKCR can be used to convince a verifier who has $\alpha_i$’s and $x_i$’s, that the prover <em>knows</em> $w_i$’s such that:</p> $\alpha_i = w_i^{x_i},\ \text{for each}\ i\in[n]$ <p>Importantly, this protocol requires that the $x_i$’s are pairwise co-prime:</p> $\gcd(x_i, x_j) = 1,\forall i,j\in[n], i\ne j$ <p>To prove knowledge of the $w_i$’s, the prover simply gives the verifier:</p> $W=\prod_{i\in [n]} w_i$ <p>To verify knowledge of $w_i$’s, the verifier (who has $\alpha_i$’s and $x_i$’s) computes $$x^* = \prod_{i\in[n]} x_i$$ and checks if:</p> $W^{x^*} \stackrel{?}{=} \prod_{i\in [n]} \alpha_i^{x^*/x_i}$ <p>The trick for the verifier is to do this computation efficiently, since the right-hand side (RHS) involves $n$ exponentiations, each of size $O(\ell n)$ bits. If done naively, this would take $O(\ell n^2)\ \Gho$ operations. Fortunately, Boneh et al.<sup id="fnref:BBF18:4" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">3</a></sup> give an $O(\ell n\log{n})$ time algorithm to compute this RHS denoted by:</p> $\multirootexp((\alpha_i, x_i)_{i\in [n]}) = \prod_{i\in [n]} \alpha_i^{x^*/x_i}$ <p>We refer you to Figure 1 in our paper<sup id="fnref:TXN20e:1" role="doc-noteref"><a href="#fn:TXN20e" class="footnote" rel="footnote">1</a></sup> for the $\multirootexp$ algorithm, which simply leverages the recursive nature of the problem. In fact, the algorithm recurses in a manner very similar to <a href="/2020/11/24/RSA-Accumulators.html#precomputing-all-membership-witnesses-fast">$\rootfactor$</a>.</p> <p>Importantly, Boneh et al. give an <em>extractor</em> that the PoKCR verifier can use to actually recover the $w_i$’s from the $x_i$’s, $\alpha_i$’s and $W$. This is what makes the protocol a proof of <em>knowledge</em>. One of our contributions is speeding up the extraction of <em>all</em> $w_i$’s from $O(\ell n^2\log{n})\ \Gho$ operations down to $O(\ell n\log^2{n})$<sup id="fnref:obs2" role="doc-noteref"><a href="#fn:obs2" class="footnote" rel="footnote">9</a></sup>. For this, we refer you to our full paper<sup id="fnref:TXN20e:2" role="doc-noteref"><a href="#fn:TXN20e" class="footnote" rel="footnote">1</a></sup>.</p> <h4 id="using-pokcr-for-incrementally-cross-aggregating-lookup-proofs">Using PoKCR for incrementally cross-aggregating lookup proofs</h4> <p>Suppose we have a lookup proof $\pi_i$ for a set of keys $K_i$ in a dictionary $D_i$ with digest $d_i = (A_i, c_i)$, where $A_i$ is the RSA accumulator over all keys in the dictionary and $c_i$ is the multi-exponentiation of RSA witnesses (i.e., the part of the proof previously denoted using $\Lambda$). Note we are changing notation slightly for ease of presentation.</p> <p>The main observation is that we can aggregate several proofs $\pi_i = (W_i, \Lambda_i)$ w.r.t. different digests $d_i$ via PoKCR because $W_i$ and $\Lambda_i$ are actually prime roots of certain group elements. To see this, recall from the <a href="#preliminaries">preliminaries</a> that:</p> <p>\begin{align} W_i &amp;= A_i^{1/e_{K_i}}\\<br /> \Lambda_i &amp;= \left(\prod_{(k,v)\in D_i\setminus K_i} (A_i^{1/e_k})^{v}\right)^{1/e_{K_i}} \end{align}</p> <p>Clearly, $W_i$ is an $e_{K_i}$-th root of $A_i$, which the verifier has. But what about $\Lambda_i$? Let $v_k$ be the value of each $k\in K_i$ and rewrite $\Lambda_i$ as: \begin{align} \Lambda_i &amp;= \left(\prod_{(k,v)\in D_i\setminus K_i} (A_i^{1/e_k})^{v}\right)^{1/e_{K_i}}\\<br /> &amp;= \left(\frac{\prod_{(k,v)\in D_i} (A_i^{1/e_k})^{v}}{\prod_{k\in K_i} (A_i^{1/e_k})^{v_k}}\right)^{1/e_{K_i}}\\<br /> &amp;= \left(c_i / \prod_{k\in K_i} (A_i^{1/e_k})^{v_k}\right)^{1/e_{K_i}}\\<br /> \end{align} Thus, if we let $$\alpha_i = c_i / \prod_{k\in K_i} (A_i^{1/e_k})^{v_k}$$, then $\Lambda_i$ is an $e_{K_i}$-th root of $\alpha_i$. Note that the verifier can compute $\alpha_i$ from $c_i, W_i$ and $K_i$ (as we describe later).</p> <p>To summarize, we have $m$ proofs $\pi_i = (W_i, \Lambda_i)$ each w.r.t. its own $d_i = (A_i, c_i)$ such that, for all $i\in [m]$: \begin{align} W_i^{e_{K_i}} &amp;= A_i\\<br /> \Lambda_i^{e_{K_i}} &amp;= \alpha_i \end{align}</p> <p>We are almost ready to aggregate with PoKCR, but we cannot yet. This is because the $e_{K_i}$’s must be pairwise co-prime for PoKCR to work! However, this is not necessarily the case, since we could have a key $k$ that is both in $K_i$ and in $K_j$ which means $e_{K_i}$ and $e_{K_j}$ will have a common factor $e_k = H(k)$.</p> <p>Fortunately, we can quickly work around this by using a different hash function $H_i$ for each dictionary $D_i$. This way, the prime representatives for $k\in K_i$ are computed as $e_k = H_i(k)$, while the prime representatives for $k\in K_j$ are computed as $e_k = H_j(k)$. As long as one cannot find any pair $(k,k’)$ with $H_i(k) = H_j(k’)$, all the $e_{K_i}$’s will be pairwise co-prime. This means we can aggregate all $m$ proofs as: \begin{align} W &amp;= \prod_{i\in[m]} W_i\\<br /> \Lambda &amp;= \prod_{i\in [m]} \Lambda_i \end{align}</p> <p>Importantly, we can do this aggregation <em>incrementally</em>: whenever a new proof arrives, we simply multiply it in the previously cross-aggregated proof.</p> <h4 id="verifying-cross-aggregated-lookup-proofs">Verifying cross-aggregated lookup proofs</h4> <p>Suppose a verifier gets a cross-aggregated proof $\pi = (W,\Lambda)$ for a bunch of $K_i$’s w.r.t. their own $d_i = (A_i, c_i),\forall i\in[m]$. How can he verify $\pi$? First, the verifier checks the PoKCR that, for each $i\in[m]$, there exists $W_i$ such that $A_i = W_i^{e_{K_i}}$:</p> $W^{e^*} \stackrel{?}{=} \multirootexp((A_i, e_{K_i})_{i\in [m]}) = \prod_{i\in [m]} A_i^{e^*/e_{K_i}}$ <p>Here, $e^*=\prod_{i\in[m]} e_{K_i}$ and each $e_{K_i} = \prod_{k\in K_i} H_i(k)$. Importantly, the verifier can recover the $W_i$’s using the PoKCR extractor (see Section 3.1 in our full paper<sup id="fnref:TXN20e:3" role="doc-noteref"><a href="#fn:TXN20e" class="footnote" rel="footnote">1</a></sup>).</p> <p>Second, the verifier checks the PoKCR for each $\alpha_i = \Lambda_i^{e_{K_i}}$. For this, the verifier must first compute each $\alpha_i = c_i / \prod_{k\in K_i} (A_i^{1/e_k})^{v_k}$, where $v_k$ is the value of each $k \in K_i$ and $e_k = H_i(k)$. The difficult part is computing all $A_i^{1/e_k}$’s, but this can be done via $\rootfactor(W_i, (e_k)_{k\in K_i})$. Once the verifier has the $\alpha_i$’s, he can check:</p> $\Lambda^{e^*} \stackrel{?}{=} \multirootexp((\alpha_i, e_{K_i})_{i\in [m]}) = \prod_{i\in [m]} \alpha_i^{e^*/e_{K_i}}$ <p>If both PoKCR checks pass, then the verifier is assured the proof verifies. Not only that, but the verifier can also disaggregate the cross-aggregated proof as we explain next.</p> <h4 id="disaggregating-cross-aggregated-proofs">Disaggregating cross-aggregated proofs</h4> <p>Since the cross-aggregated proof $\pi = (W,\Lambda)$ is a PoKCR proof, this mean the PoKCR extractor can be used to recover the original proofs $(\pi_i)_{i\in[m]}$ that $\pi$ was aggregated from. How?</p> <p>Well, we already showed how the verifier must extract the $W_i$’s in the original proofs, which he needs for reconstructing the $\alpha_i$’s to verify the $\Lambda$ part of the cross-aggregated proof. In a similar fashion, the verifier can also extract all the $\Lambda_i$’s aggregated in $\Lambda$. This way, the verifier can recover the original proofs. Note that this implies cross-aggregated proofs are <em>updatable</em> by:</p> <ol> <li>Cross-disaggregating them into the original lookup proofs,</li> <li>Updating these lookup proofs,</li> <li>And cross-reaggregating them back.</li> </ol> <h2 id="conclusion">Conclusion</h2> <p>To conclude, we show that generalizing CF to a larger key-space results in a versatile <em>authenticated dictionary (AD)</em> scheme that supports updating proofs and digests and supports aggregating proofs across different dictionaries in an incremental fashion. In a future post, we strengthen the security of this construction, which makes it applicable to more adversarial applications such as transparency logging. As always, see our full paper for details<sup id="fnref:TXN20e:4" role="doc-noteref"><a href="#fn:TXN20e" class="footnote" rel="footnote">1</a></sup>.</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:TXN20e" role="doc-endnote"> <p><strong>Authenticated Dictionaries with Cross-Incremental Proof (Dis)aggregation</strong>, by Alin Tomescu and Yu Xia and Zachary Newman, <em>in Cryptology ePrint Archive, Report 2020/1239</em>, 2020, <a href="https://eprint.iacr.org/2020/1239">[URL]</a> <a href="#fnref:TXN20e" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:TXN20e:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a> <a href="#fnref:TXN20e:2" class="reversefootnote" role="doc-backlink">&#8617;<sup>3</sup></a> <a href="#fnref:TXN20e:3" class="reversefootnote" role="doc-backlink">&#8617;<sup>4</sup></a> <a href="#fnref:TXN20e:4" class="reversefootnote" role="doc-backlink">&#8617;<sup>5</sup></a></p> </li> <li id="fn:SSY01" role="doc-endnote"> <p><strong>Blind, Auditable Membership Proofs</strong>, by Sander, Tomas and Ta-Shma, Amnon and Yung, Moti, <em>in Financial Cryptography</em>, 2001 <a href="#fnref:SSY01" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:BBF18" role="doc-endnote"> <p><strong>Batching Techniques for Accumulators with Applications to IOPs and Stateless Blockchains</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/1188</em>, 2018, <a href="https://eprint.iacr.org/2018/1188">[URL]</a> <a href="#fnref:BBF18" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:BBF18:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a> <a href="#fnref:BBF18:2" class="reversefootnote" role="doc-backlink">&#8617;<sup>3</sup></a> <a href="#fnref:BBF18:3" class="reversefootnote" role="doc-backlink">&#8617;<sup>4</sup></a> <a href="#fnref:BBF18:4" class="reversefootnote" role="doc-backlink">&#8617;<sup>5</sup></a></p> </li> <li id="fn:LK15" role="doc-endnote"> <p><strong>Revocation Transparency</strong>, by Ben Laurie and Emilia Kasper, 2015, <a href="https://www.links.org/files/RevocationTransparency.pdf">[URL]</a> <a href="#fnref:LK15" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:obs1" role="doc-endnote"> <p>We were not the first to make this observation; see the work by Agrawal and Raghuraman<sup id="fnref:AR20" role="doc-noteref"><a href="#fn:AR20" class="footnote" rel="footnote">10</a></sup> <a href="#fnref:obs1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Feis20Multi" role="doc-endnote"> <p><strong>Multi-layer hashmaps for state storage</strong>, by Dankrad Feist, 2020, <a href="https://ethresear.ch/t/multi-layer-hashmaps-for-state-storage/7211/print">[URL]</a> <a href="#fnref:Feis20Multi" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:CF13e" role="doc-endnote"> <p><strong>Vector Commitments and their Applications</strong>, by Dario Catalano and Dario Fiore, <em>in Cryptology ePrint Archive, Report 2011/495</em>, 2011, <a href="https://eprint.iacr.org/2011/495">[URL]</a> <a href="#fnref:CF13e" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:GRWZ20e" role="doc-endnote"> <p><strong>Pointproofs: Aggregating Proofs for Multiple Vector Commitments</strong>, by Sergey Gorbunov and Leonid Reyzin and Hoeteck Wee and Zhenfei Zhang, 2020, <a href="https://eprint.iacr.org/2020/419">[URL]</a> <a href="#fnref:GRWZ20e" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:GRWZ20e:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a></p> </li> <li id="fn:obs2" role="doc-endnote"> <p>See Section 3.1 in our full paper<sup id="fnref:TXN20e:5" role="doc-noteref"><a href="#fn:TXN20e" class="footnote" rel="footnote">1</a></sup> <a href="#fnref:obs2" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:AR20" role="doc-endnote"> <p><strong>KVaC: Key-Value Commitments for Blockchains and Beyond</strong>, by Shashank Agrawal and Srinivasan Raghuraman, <em>in Cryptology ePrint Archive, Report 2020/1161</em>, 2020, <a href="https://eprint.iacr.org/2020/1161">[URL]</a> <a href="#fnref:AR20" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div>Alin Tomescutl;dr: We build an authenticated dictionary (AD) from Catalano Fiore vector commitments that has constant-sized, aggregatable proofs and supports a stronger notion of cross-incremental proof disaggregation. Our AD could be used for stateless validation in cryptocurrencies with smart contract execution. In a future post, we will extend this AD with stronger security, non-membership proofs and append-only proofs, which makes it applicable to transparency logging. This is joint work with my brilliant (ex-)labmates from MIT, Alex (Yu) Xia and Zack Newman. Authenticated dictionaries (ADs) are an important cryptographic primitive which lies at the core of cryptocurrencies such as Ethereum and of transparency logs such as Certificate Transparency (CT). Typically, ADs are constructed by Merkleizing a lexicographically-ordered data structure such as a binary search tree, a prefix tree or a skip list. However, our work takes a different, more algebraic direction, building upon the Catalano-Fiore (CF) vector commitment (VC) scheme. This has the advantage of giving us constant-sized proofs which are updatable and aggregatable, with a novel notion of cross-incrementality. Importantly, this combination of feautres is not supported by Merkle trees or any other previous VC scheme.Catalano-Fiore Vector Commitments2020-11-24T00:00:00+00:002020-11-24T00:00:00+00:00https://alinush.github.io//2020/11/24/Catalano-Fiore-Vector-Commitments<!-- TODO: Write an intro paragraph here --> <p>A <em>vector commitment (VC)</em> scheme allows a <strong>prover</strong> with access to a vector $\mathbf{v} = [ v_1, \dots, v_n ]$ to convince any <strong>verifier</strong> that position $i$ in $\mathbf{v}$ stores $v_i$ for any index $i\in[n]$. Importantly, verifers only store a succinct <strong>digest</strong> of the vector (e.g., a 32-byte hash) rather than the full vector $\mathbf{v}$.</p> <!--more--> <p hidden="">$$\def\Adv{\mathcal{A}} \def\Badv{\mathcal{B}} \def\GenGho{\mathsf{GenGroup}_?} \def\Ghosz{|\Gho|} \def\Ghoid{1_{\Gho}} \def\multirootexp{\mathsf{MultiRootExp}} \def\primes{\mathsf{Primes}} \def\QRn{\mathsf{QR}_N} \def\rootfactor{\mathsf{RootFactor}} \def\shamirtrick{\mathsf{ShamirTrick}} \def\vect#1{\mathbf{#1}}$$</p> <p>Below, we review <em>Catalano and Fiore’s</em> elegant VC scheme<sup id="fnref:CF13e" role="doc-noteref"><a href="#fn:CF13e" class="footnote" rel="footnote">1</a></sup> built from hidden-order groups, extended with the enhancements proposed by Lai and Malavolta<sup id="fnref:LM18" role="doc-noteref"><a href="#fn:LM18" class="footnote" rel="footnote">2</a></sup> and by Campanelli et al.<sup id="fnref:CFGplus20e" role="doc-noteref"><a href="#fn:CFGplus20e" class="footnote" rel="footnote">3</a></sup> We also make one new, small observation: <a href="#disaggregating-proofs">proof disaggregation</a> in this VC is faster than originally thought, which helps precompute all proofs slightly faster.</p> <h2 id="preliminaries">Preliminaries</h2> <p>Let $[n] = \{1,2,\dots, n\}$ and $[a,b] = \{a, a+1,\dots, b-1, b\}$.</p> <p>We assume the reader is familiar with <a href="/2020/11/24/RSA-accumulators.html">RSA accumulators</a>, which Catalano-Fiore (CF) VCs rely heavily upon. Also, we assume familiarity with the <a href="/2020/11/24/RSA-accumulators.html#precomputing-all-membership-witnesses-fast">$\rootfactor$ algorithm</a> for computing <strong>all</strong> $e_j$th roots of $g^{\prod_{i\in[n]} e_i}, j\in [n]$.</p> <h3 id="shamirs-trick">Shamir’s trick</h3> <p>Given $z^{1/a}$ and $z^{1/b}$ where $z\in\Gho$ and $a,b\in\Z$ with $\gcd(a,b) = 1$, one can compute $z^{1/ab} = (z^{1/a})^y (z^{1/b})^x$ where $x,y$ are the Bezout coefficients for $a,b$ such that $ax+by=1$.</p> <p>This is because:</p> $(z^{1/a})^y (z^{1/b})^x = z^\frac{by}{ab} z^\frac{ax}{ab} = z^\frac{ax+by}{ab} = z^\frac{1}{ab}$ <p>We often use $z^\frac{1}{ab}\leftarrow \shamirtrick(z^{1/a}, z^{1/b}, a, b)$ to denote performing a Shamir’s trick..</p> <p>Computing $z^{1/(ab)}$ takes $$O(\max(\vert x\vert,\vert y\vert))\ \Gho$$ operations since the Bezout coefficients $(a,b)$ are as big as $(x,y)$. (Some extra time is needed to compute the Bezout coefficients, but we ignore it here.)</p> <h2 id="public-parameters">Public parameters</h2> <p>To set up the VC scheme, a hidden-order group $\Gho$ with generator $g$ must be picked such that nobody knows the order of the group. Importantly, the RSA problem needs to be hard in this group. <!-- TODO: reference assumptions --> Typically, $\Gho = \ZNs$ where $N=pq$ is picked via a secure <em>multi-party computation (MPC) ceremony</em> such that nobody knows the factorization of $N$ (and thus nobody knows the order of $\Gho$; i.e., $\vert\Gho\vert = \phi(N)= (p-1)(q-1)$).</p> <p>Then, a collision-resistant hash function $H : [n] \rightarrow \primes^{\ell+1}$ must be fixe. $H$ will map each vector index $i\in [n]$ to a prime $e_i = H(i)$ such that $2^\ell &lt; e_i &lt; 2^{\ell+1}$ where $\ell$ is the maximum size in bits of a vector element $v_i$. We often use $e_K = \prod_{k\in K} e_k$ to denote a product of a subset of such primes, where $K\subseteq[n]$.</p> <h2 id="digest-or-commitments">Digest (or commitments)</h2> <p>Instead of commitments, we’ll stick to the <em>digest</em> terminology.</p> <p>The digest of a vector $\vect{v} = [ v_1, \dots, v_n ]$ consists of two parts. First, an <a href="/2020/11/24/RSA-accumulators.html">RSA accumulator</a> over all positions in the vector: \begin{align} S &amp;= g^{\prod_{i\in[n]} e_i} \end{align}</p> <p>Second, a multi-exponentiation of each <a href="/2020/11/24/RSA-accumulators.html#membership-witnesses">RSA membership witness</a> (w.r.t. $S$) for position $i$ to the value $v_i$: \begin{align} \Lambda &amp;= \prod_{i\in [n]} (S^{1/e_i})^{v_i} \end{align}</p> <p>We often use $d(\vect{v}) = (S, \Lambda)$ to denote the digest of a vector.</p> <p class="info">Computing the digest requires computing all $S^{1/e_i}, \forall i\in[n]$. As described <a href="/2020/11/24/RSA-accumulators.html#precomputing-all-membership-witnesses-fast">before</a>, this can be done in $O(n\log{n})$ group exponentiations via $$(S^{1/e_i})_{i\in[n]} \leftarrow \rootfactor(g, (e_i)_{i\in[n]})$$. Since each exponentiation is by an $(\ell+1)$-bit prime $e_i$, this takes $O(\ell n\log{n})\ \Gho$ operations. Then, $\Lambda$ can be computed with an additional $O(\ell n)\ \Gho$ operations.</p> <p>The beauty of CF VCs is that, once you understand what a digest looks like, everything follows very naturally.</p> <h2 id="proofs-are-just-digests">Proofs are just digests</h2> <p>At minimum, in any VC scheme, a verifier can be convinced of the value $v_i$ of any position $i$ in the vector using a <strong>proof</strong> $\pi_i$. To convince the verifier of multiple values $(\vect{v}_i)_{i\in I}$, this would require individual proofs for each position $i\in I$. However, in VCs like CF, a single $I$-subvector proof $\pi_I$ can be constructed for all $(v_i)_{i\in I}$. Importantly, the size of $\pi_I$ is always constant, independent of $\vert I\vert$.</p> <p>In CF, an $I$-subvector proof $\pi_I$ is just the digest of the vector $\vect{v}$ but <em>“without”</em> positions $I$ in it. We abuse notation and denote this by $$\vect{v} \setminus I = (v_i)_{i\in[n] \setminus I}$$. Thus, $\pi_I = d(\vect{v}\setminus I)$.</p> <p>Let $e_I = \prod_{i\in I} e_i$. Then, the proof $\pi_I$ is just: \begin{align} S_I &amp;= S^{1/e_I} = g^{\prod_{i\in[n]\setminus I} e_i}\\<br /> \Lambda_I &amp;= \prod_{i\in [n]\setminus I} (S_I^{1/e_i})^{v_i} \end{align}</p> <p>Observe that the accumulator $S_I$ is over all positions $i$ except the ones in $I$. Also, observe that the membership witnesses in $\Lambda_I$ are w.r.t. to $S_I$ (not $S$) and that $\Lambda_I$ does not contain witnesses for $i\in I$. This is what it means to commit to a vector $\vect{v}$ “without” positions $I$ in it.</p> <p class="info">Computing $\pi_I$ takes $O(\ell(n-\vert I \vert)\log(n-\vert I\vert))\ \Gho$ operations, dominated by the cost to compute $$(S_I^{1/e_i})_{i\in I}$$ via $$\rootfactor(g, (e_i)_{i\in[n]\setminus I})$$.</p> <p>To verify the proof, one simply “adds back” $$\vect{v}_I$$ to $$d(\vect{v}\setminus I) = \pi_I$$ and checks it obtains the digest $d(\vect{v}) = (S, \Lambda)$ of the vector: \begin{align} S &amp;\stackrel{?}{=} S_I^{e_I}\\<br /> \Lambda &amp;\stackrel{?}{=} \Lambda_I^{e_I} \prod_{i\in I} (S^{1/e_i})^{v_i} \end{align}</p> <p class="warning">One problem is that all $$(S^{1/e_i})_{i\in I}$$ must be computed via $$\rootfactor(S_I, (e_i)_{i\in I})$$, which would take $O(\ell \vert I\vert \log\vert I\vert)\ \Gho$ operations. We explain <a href="#updating-digest">next</a> how this can be done in $O(\ell \vert I \vert)\ \Gho$ operations by adding each $v_i, i\in I$ back into $\pi_I$ <em>sequentially</em>!</p> <h2 id="updating-digest">Updating digest</h2> <p>It is possible to update the digest $d(\vect{v})=(S,\Lambda)$ after an element $v_i$ changes to $v_i + \delta_i$. For this, the RSA membership witness $S^{1/e_i}$ is needed as helper information, which we refer to as an <strong>update key</strong>.</p> <p>Since $S$ does not contain any information about $v_i$ (just about $i$), $S$ stays the same. However, $\Lambda$ must change into $\Lambda’$ as follows: \begin{align} \Lambda’ = \Lambda \cdot (S^{1/e_i})^{\delta_i} \end{align}</p> <p class="info">This technique can be generalized to work with many positions $i\in I$ changing by $\delta_i$. This either requires all the $S^{1/e_i}$ update keys or an <strong>aggregated update key</strong> $$S_I = S^{1/e_I} = S^{\prod_{i\in I} 1/e_i}$$ as auxiliary information. In this last case, all the $$(S^{1/e_i})_{i\in I} \leftarrow \rootfactor(S_I, (e_i)_{i\in I})$$ can be computed in $O(\ell \vert I \vert \log \vert I \vert)\ \Gho$ operations from the aggregated update key. Then the update can be performed as $$\Lambda' = \Lambda \cdot \prod_{i\in I} (S^{1/e_i})^{\delta_i}$$</p> <h3 id="adding-new-positions-to-the-vector">Adding new positions to the vector</h3> <p>It is also possible to “extend” the vector $\vect{v}$ of size $n$ to size $n+1$ by adding an extra element $v_{n+1}$. For this, the digest can be updated as: \begin{align} S’ &amp;= S^{e_{n+1}}\\<br /> \Lambda’ &amp;= \Lambda^{e_{n+1}} S^{v_{n+1}} = \dots = \prod_{i\in [n+1]} (S’^{1/e_i})^{v_i} \end{align}</p> <p class="info">This can be generalized to adding $\Delta$ new positions, and would take $O(\ell \Delta)\ \Gho$ operations by applying each extension sequentially. In fact, this how one would verify a subvector proof fast: by adding back the elements being verified to the proof (which can be viewed as a digest) and checking the actual digest is obtained!</p> <h2 id="updating-proofs">Updating proofs</h2> <p>Since proofs are just digests they are, in principle, updatable in the same fashion, with a few extra details.</p> <p>Suppose we have a proof $\pi_I$ for a subvector $$\vect{v}_I$$: \begin{align} S_I &amp;= S^{1/e_I} = S^{1/\prod_{i\in I} e_i} = g^{\prod_{i\in[n]\setminus I} e_i}\\<br /> \Lambda_I &amp;= \prod_{i\in [n]\setminus I} (S_I^{1/e_i})^{v_i} \end{align}</p> <h3 id="case-1-iin-i-changed">Case 1: $i\in I$ changed</h3> <p>First, suppose a position $i\in I$ changes from $$v_i$$ to $$v_i + \delta_i$$. Then, the proof $\pi_I$ remains the same! This is because the proof contains no information about $$v_i$$, since the proof is the digest of $\vect{v}$ without any positions $i\in I$.</p> <h3 id="case-2-jnotin-i-changed">Case 2: $j\notin I$ changed</h3> <p>Second, suppose a position $j\notin I$ changes from $$v_j$$ to $$v_j+\delta_j$$. In this case, we only need to change $\Lambda_I$ into a $\Lambda_I’$ such that: \begin{align} \Lambda_I’ = \Lambda_I \cdot (S_I^{1/e_j})^{\delta_j} \end{align}</p> <p>The difficult part is computing $S_I^{1/e_j}$. We could do it from scratch but that would take $O(\ell (n - \vert I \vert))\ \Gho$ operations. Instead, as with the <a href="#updating-digest">digest update</a>, we assume we are given the $S^{1/e_j}$ update key associated with the changed position $j$. Fortunately, we know that we can compute $S_I^{1/e_j} = S^{1/(e_I e_j)}$ using <a href="#shamirs-trick">Shamir’s trick</a> on $S_I = S^{1/e_I}$ and $S^{1/e_j}$! (This takes $O(\ell \vert I \vert)\ \Gho$ operations which is faster when $|I|$ is much smaller than $n$.)</p> <p class="info">This technique can also be generalized to updating $\pi_I$ after many positions $j\in J$ changed by $\delta_j$. (Assume without loss of generality that $J\cap I =\varnothing$ or let $J \leftarrow J \setminus I$ otherwise.) This either requires all $S_I^{1/e_j}$ update keys or the <em>aggregated update key</em> $$S_{I\cup J} = S^{1/(e_I e_J)} = S^{\prod_{k\in I\cup J} 1/e_k}$$ as auxiliary information. In this last case, all the $$(S_I^{1/e_j})_{j\in J} \leftarrow \rootfactor(S_{I\cup J}, (e_j)_{j\in J})$$ can be computed in $O(\ell \vert J \vert \log \vert J \vert)\ \Gho$ operations from the aggregated update key. Then, $$\Lambda_I' = \Lambda_I \cdot \prod_{j\in J} (S_I^{1/e_j})^{\delta_j}$$.</p> <h3 id="case-3-adding-new-positions">Case 3: Adding new positions</h3> <p>Third, suppose a new position $v_{n+1}$ was added to the vector. Then the proof update proceeds similar to the <a href="#adding-new-positions-to-the-vector">digest update</a> for adding new positions: \begin{align} S_I’ &amp;= S_I^{e_{n+1}}\\<br /> \Lambda_I’ &amp;= \Lambda_I^{e_{n+1}} S_I^{v_{n+1}} = \dots = \prod_{i\in [n+1] \setminus I} (S_I’^{1/e_i})^{v_i} \end{align}</p> <h2 id="disaggregating-proofs">Disaggregating proofs</h2> <p>Suppose we want to <strong>disaggregate</strong> a proof $\pi_I=(S_I,\Lambda_I)$ for $\vect{v}_I$ into a proof $\pi_K=(S_K,\Lambda_K)$ for $\vect{v}_K$, where $K\subset I$.</p> <p>In other words, we want to go from $d(\vect{v}\setminus I)$ to $d(\vect{v}\setminus K)$. Since $K\subset I$ we can write $K = I\setminus \Delta$ for some set $\Delta$ of indices. <!-- Thus, slightly abusing notation, $(\vect{v} \setminus K) = (\vect{v} \setminus (I \setminus \Delta)) = (\vect{v} \setminus I \cup \Delta)$ --> Thus, all we need to do is add back the elements from $\Delta$ to $\pi_I$.</p> <p>So, for each $i\in \Delta$, we simply add back $$(i,v_i)$$ to $$\pi_I$$ (as explained <a href="#updating-digest">here</a>) and obtain $$\pi_K$$. This takes $O(\ell\vert\Delta\vert)$ $\Gho$ operations.</p> <h2 id="aggregating-proofs">Aggregating proofs</h2> <p>To aggregate two subvector proofs for $$\vect{v}_I$$ and $$\vect{v}_J$$, things are bit more involved. Let $$\pi_I = (S_I,\Lambda_I)$$ and $$\pi_J = (S_J,\Lambda_J)$$ denote the two proofs. Assume that $$I\cap J = \varnothing$$ and if not, just set $$J = J\setminus I$$ and disaggregate $$\pi_J$$ into $$\pi_{J\setminus I}$$. (Can also set $$I=I\setminus J$$ and disaggregate $$\pi_I$$ instead.)</p> <p>Denote the aggregated proof by $$\pi_{I\cup J} = (S_{I \cup J}, \Lambda_{I \cup J})$$. We can easily compute the first part of the proof as:</p> $S_{I\cup J} = \shamirtrick(S_I, S_J, e_I, e_J) = S^{1/(e_I e_J)}$ <p>For the second part, focusing on $\Lambda_I$, recall that: \begin{align} \Lambda_I &amp;= \prod_{i\in [n]\setminus I} (S_I^{1/e_i})^{v_i} \end{align} We will tweak it as $$\Lambda_I^*$$ by removing all elements $v_j, j\in J$ from $$\Lambda_I$$: \begin{align} \Lambda_I^* &amp;= \Lambda_I / \prod_{j\in J} (S_I^{1/e_j})^{v_j}\\<br /> &amp;= \prod_{i\in [n]\setminus (I\cup J)} (S_I^{1/e_i})^{v_i} \end{align} But for this, we need to compute $$S_I^{1/e_j},\forall j\in J$$ via $\rootfactor$: \begin{align} (S_I^{1/e_j})_{j\in J} \leftarrow \rootfactor(S_{I\cup J}, (e_j)_{j\in J}) \end{align} In a similar, fashion, we can compute \begin{align} \Lambda_J^* &amp;= \prod_{i\in [n]\setminus (I\cup J)} (S_J^{1/e_i})^{v_i} \end{align} Next, observe that $$\Lambda_I^*$$ and $$\Lambda_J^*$$ can be rewritten as: \begin{align} \Lambda_I^* &amp;= \left(\prod_{i\in [n]\setminus (I\cup J)} (S^{1/e_i})^{v_i}\right)^{1/e_I}\\<br /> \Lambda_J^* &amp;= \left(\prod_{i\in [n]\setminus (I\cup J)} (S^{1/e_i})^{v_i}\right)^{1/e_J} \end{align} Thus, if we apply a Shamir trick on them, we can obtain $\Lambda_{I\cup J}$: \begin{align} \Lambda_{I\cup J} &amp;= \shamirtrick(\Lambda_I^*, \Lambda_J^*, e_I, e_J)\\<br /> &amp;= \left(\prod_{i\in [n]\setminus (I\cup J)} (S^{1/e_i})^{v_i}\right)^{1/(e_I e_J)}\\<br /> &amp;= \prod_{i\in [n]\setminus (I\cup J)} (S_{I\cup J}^{1/e_i})^{v_i} \end{align}</p> <p class="info">Overall, if $b=\max(\vert I\vert, \vert J \vert)$, this requires Shamir tricks of size $b$ and $\rootfactor$’s of size $b$, so it takes $O(\ell b\log{b})\ \Gho$ operations.</p> <h2 id="precomputing-all-proofs-via-disaggregation">Precomputing all proofs via disaggregation</h2> <p>Campanelli et al.<sup id="fnref:CFGplus20e:1" role="doc-noteref"><a href="#fn:CFGplus20e" class="footnote" rel="footnote">3</a></sup> explain how to compute all proofs fast via disaggregation. The idea is to start with a proof for the whole vector $\vect{v}$ and then disaggregate it into a proof for the left half of the vector and the right half. Then, one repeats recursively on these two halves. This determines a tree of subvector proofs, where the leaves store the proofs for the individual vector and each internal node stores an aggregation of its children’s proofs (with the root storing the proof for the full vector).</p> <!-- NOTE: This was the LaTeX for the tree. Just be sure to add \usepackage{forest} in the preamble in OS X's "LaTeX it!" if you redo this. \begin{figure*}[t] { \normalsize \begin{center} \begin{forest} for tree={ % fit=band,% spaces the tree out a little to avoid collisions % fit=tight,% spaces the tree out less % fit=rectangle, inner sep=4, } [{$\pi_{[1,8]},\{v_1,\dots,v_8\}$} [{$\pi_{[1,4]},\{v_1,\dots,v_4\}$} [{$\pi_{[1,2]},\{v_1,v_2\}$} [{$\pi_1,\{v_1\}$} [, no edge, tier=odd ] ] [{$\pi_2,\{v_2\}$} , tier=odd ] ] [{$\pi_{[3,4]},\{v_3,v_4\}$} [{$\pi_3,\{v_3\}$} [, no edge, tier=odd ] ] [{$\pi_4,\{v_4\}$} , tier=odd ] ] ] [{$\pi_{[5,8]},\{v_5,\dots,v_8\}$} [{$\pi_{[5,6]},\{v_5,v_6\}$} [{$\pi_5,\{v_5\}$} [, no edge, tier=odd ] ] [{$\pi_6,\{v_6\}$} , tier=odd ] ] [{$\pi_{[7,8]},\{v_7,v_8\}$} [{$\pi_7,\{v_7\}$} [, no edge, tier=odd ] ] [{$\pi_8,\{v_8\}$} , tier=odd ] ] ] ] \end{forest} \end{center} } \end{figure*} --> <p>Here’s an example for $n=8$:</p> <div align="center"><img style="width:95%" src="/pictures/cf-proof-precomp.png" /></div> <p>Looked at it differently, clearly one can compute this tree via proof aggregation, which we discuss <a href="#aggregating-proofs">next</a>, by starting with the leaves storing individual proofs for each position $i$ and aggregating up the tree. The key observation is that one can also compute this tree by starting with the root, which stores the proof for the full vector (i.e., the empty digest), <em>disaggregating</em> this proof into two halves, and recursing on these two halves.</p> <p class="info">This very much resembles the recursion tree used to implement $\rootfactor$ efficiently (see <a href="/2020/11/24/RSA-accumulators.html#precomputing-all-membership-witnesses-fast">here</a>). Indeed, if you think about it, this tree implicitly computes the $\rootfactor$ tree too, since each CF proof for $v_i$ has an RSA membership witness for $i$ in it.</p> <p>Note that Campanelli et al.<sup id="fnref:CFGplus20e:2" role="doc-noteref"><a href="#fn:CFGplus20e" class="footnote" rel="footnote">3</a></sup> claim $O(\ell n\log^2{n})\ \Gho$ operations for this algorithm, but the faster disaggregation explained above actually gives $O(\ell n\log{n})$.</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:CF13e" role="doc-endnote"> <p><strong>Vector Commitments and their Applications</strong>, by Dario Catalano and Dario Fiore, <em>in Cryptology ePrint Archive, Report 2011/495</em>, 2011, <a href="https://eprint.iacr.org/2011/495">[URL]</a> <a href="#fnref:CF13e" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:LM18" role="doc-endnote"> <p><strong>Subvector Commitments with Application to Succinct Arguments</strong>, by Russell W.F. Lai and Giulio Malavolta, <em>in Cryptology ePrint Archive, Report 2018/705</em>, 2018, <a href="https://eprint.iacr.org/2018/705">[URL]</a> <a href="#fnref:LM18" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:CFGplus20e" role="doc-endnote"> <p><strong>Vector Commitment Techniques and Applications to Verifiable Decentralized Storage</strong>, by Matteo Campanelli and Dario Fiore and Nicola Greco and Dimitris Kolonelos and Luca Nizzardo, 2020, <a href="https://eprint.iacr.org/2020/149">[URL]</a> <a href="#fnref:CFGplus20e" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:CFGplus20e:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a> <a href="#fnref:CFGplus20e:2" class="reversefootnote" role="doc-backlink">&#8617;<sup>3</sup></a></p> </li> </ol> </div>Alin TomescuA vector commitment (VC) scheme allows a prover with access to a vector $\mathbf{v} = [ v_1, \dots, v_n ]$ to convince any verifier that position $i$ in $\mathbf{v}$ stores $v_i$ for any index $i\in[n]$. Importantly, verifers only store a succinct digest of the vector (e.g., a 32-byte hash) rather than the full vector $\mathbf{v}$.RSA Accumulators2020-11-24T00:00:00+00:002020-11-24T00:00:00+00:00https://alinush.github.io//2020/11/24/RSA-accumulators<p>An <strong>RSA accumulator</strong> is an <em>authenticated set</em> built from cryptographic assumptions in hidden-order groups such as $\mathbb{Z}_N^*$. RSA accumulators enable a <strong>prover</strong>, who stores the full set, to convince any <strong>verifier</strong>, who only stores a succinct <strong>digest</strong> of the set, of various set relations such as (non)membership, subset or disjointness. For example, the prover can prove membership of elements in the set to verifiers who have the digest.</p> <!--more--> <p hidden="">$$\def\Adv{\mathcal{A}} \def\Badv{\mathcal{B}} \def\GenGho{\mathsf{GenGroup}_?} \def\Ghosz{|\Gho|} \def\Ghoid{1_{\Gho}} \def\primes{\mathsf{Primes}} \def\QRn{\mathsf{QR}_N} \def\multirootexp{\mathsf{MultiRootExp}} \def\rootfactor{\mathsf{RootFactor}} \def\vect#1{\mathbf{#1}}$$</p> <p>The <em>digest</em> of the set is often referred to as the <em>accumulator</em> of the set. <!-- TODO: Link to RSA assumptions post --></p> <p>RSA accumulators were introduced by <em>Benaloh and de Mare</em><sup id="fnref:Bd93" role="doc-noteref"><a href="#fn:Bd93" class="footnote" rel="footnote">1</a></sup> and later extended by <em>Li et al.</em><sup id="fnref:LLX07" role="doc-noteref"><a href="#fn:LLX07" class="footnote" rel="footnote">2</a></sup> with non-mmebership proofs. Recently, Boneh et al.<sup id="fnref:BBF18" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">3</a></sup> extended RSA accumulators with many new features.</p> <p class="warning">This post is not a full treatment of RSA accumulators, but will be extended over time.</p> <h2 id="digest-or-accumulator-of-a-set">Digest (or accumulator) of a set</h2> <p>Let $H$ be a collision-resistant hash function that maps its input to prime numbers. Let $g$ be the generator of a hidden-order group $\Gho$ where the Strong RSA problem is hard. <!-- TODO: reference assumptions --> Let $T = \{b_1, b_2, \dots, b_n\}$ and let $e_i = H(b_i)$ be the <em>prime representative</em> of $b_i$.</p> <p>The accumulator of $T$ is:</p> $a = g^{\prod_{i\in[n]} e_i}$ <p>Note that this can be computed in $O(n)$ exponentiations in $\Gho$.</p> <h2 id="membership-witnesses">Membership witnesses</h2> <p>To prove that $b_i$ is in the accumulator, a <em>membership witness</em> can be computed:</p> $w_i = g^{\prod_{j\in[n]\setminus\{i\}} e_j} = a^{1/e_i}$ <p>Note that the witness is simply the accumulator of the set $T \setminus \{b_i\}$ and is just an $e_i$th root of $a$!</p> <p class="warning">Unfortunately, computing this root cannot be done using an exponentiation by $1/e_i$ because $e_i$ cannot be inverted without knowing the order of the group $\Gho$. Instead, computing $w_i$ has to be done by exponentiating $g$ by the $n-1$ different $e_i$’s and thus takes $O(n)$ time, which can be slow. We explain <a href="#precomputing-all-membership-witnesses-fast">below</a> how this can be done in $O(\log{n})$ amortized time per witness.</p> <p>To verify the witness against the accumulator $a$, one checks if:</p> $a \stackrel{?}{=} w_i^{e_i}$ <p>In other words, one simply “adds back” $b_i$ to the set accumulated in $w_i$ and checks if the result equals $a$. This takes one exponentiation in $\Gho$.</p> <p>Membership witnesses can be easily generalized into <em>batch membership witnesses</em>. For example, if one wants to prove that all $(b_i)_{i\in I}$ are in the accumulator, they can compute:</p> $w_I = g^{\prod_{i\in[n]\setminus I} e_i} = a^{1/ \prod_{i\in I} e_i}$ <p>The verification proceeds analogously but takes $O(\vert I \vert)$ exponentiations:</p> $a \stackrel{?}{=} w_i^{\prod_{i\in I}e_i}$ <h2 id="precomputing-all-membership-witnesses-fast">Precomputing all membership witnesses fast</h2> <p>Computing all $n$ membership witnesses naively takes $O(n^2)$ exponentiations in $\Gho$, which does not scale well. Fortunately, <em>Sander et al.</em><sup id="fnref:SSY01" role="doc-noteref"><a href="#fn:SSY01" class="footnote" rel="footnote">4</a></sup> give a divide-and-conquer approach for computing all witnesses which takes $O(n\log{n})$ exponentiations in $\Gho$.</p> <p>The key observation is that half of the witnesses require computing $g^{\prod_{i\in[1,n/2]} e_i}$ while the other half require computing $g^{\prod_{i\in[n/2+1,n]} e_i}$. If done naively, these computations would be repeated many times unnecessarily. But one could compute the witnesses recursively in a tree-like fashion as follows ($n=8$ in this example):</p> <div align="center"><img style="width:100%" src="/pictures/rsa-membwit-precomp.png" /></div> <p class="info"><strong>Intuition:</strong> You can think of each node as (1) a set of elements and (2) its <em>batch membership witness</em> w.r.t. the accumulator $a$. Then, this algorithm simply splits the set into two halves and disaggregates the witness into witnesses for the two halves. This repeats until witnesses for individual elements are obtained at the bottom.</p> <p>In other words, this algorithm computes every $e_i$th root of $a$ (i.e., $g^{e_1 \dots e_{i-1} e_{i+1} \dots e_{n}}$) for all $i\in[n]$. The time to compute a tree of size $n$ leaves can be defined recursively as $T(n) = 2T(n/2) + O(n)$ because the amount of work done at the root of a size-$n$ subtree is $O(n)$ exponentiations plus the time to recurse on its two children. This gives $T(n) = O(n\log{n})$ exponentiations.</p> <p>Boneh et al.<sup id="fnref:BBF18:1" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">3</a></sup> baptised this algorithm as $$\rootfactor$$. Specifically,</p> $\rootfactor(g, (e_i)_{i\in[n]}) = (w_i)_{i\in[n]} = (a^{1/e_i})_{i\in[n]} = \left((g^{\prod_{j\in[n]} e_j})^{1/e_i}\right)_{i\in[n]}$ <!-- **TODO:** Future post: update membership, aggregate membership, non-membership witnesses + update, subset relations, disjointness relations --> <h2 id="more-to-come">More to come</h2> <p>There is so much more to be said about RSA accumulators! Hopefully, we will update this post over time with:</p> <ul> <li>Accumulator updates</li> <li>Membership witness updates</li> <li>Cross-incremental (dis)aggregation of membership witnesses<sup id="fnref:TXN20e" role="doc-noteref"><a href="#fn:TXN20e" class="footnote" rel="footnote">5</a></sup></li> <li>Non-membership witnesses <ul> <li>Updates</li> <li>Aggregation</li> </ul> </li> <li>Disjointness witnesses<sup id="fnref:Tome20" role="doc-noteref"><a href="#fn:Tome20" class="footnote" rel="footnote">6</a></sup></li> <li>Subset witnesses<sup id="fnref:Tome20:1" role="doc-noteref"><a href="#fn:Tome20" class="footnote" rel="footnote">6</a></sup></li> </ul> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:Bd93" role="doc-endnote"> <p><strong>One-Way Accumulators: A Decentralized Alternative to Digital Signatures</strong>, by Benaloh, Josh and de Mare, Michael, <em>in EUROCRYPT ‘93</em>, 1994 <a href="#fnref:Bd93" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:LLX07" role="doc-endnote"> <p><strong>Universal Accumulators with Efficient Nonmembership Proofs</strong>, by Li, Jiangtao and Li, Ninghui and Xue, Rui, <em>in Applied Cryptography and Network Security</em>, 2007 <a href="#fnref:LLX07" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:BBF18" role="doc-endnote"> <p><strong>Batching Techniques for Accumulators with Applications to IOPs and Stateless Blockchains</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/1188</em>, 2018, <a href="https://eprint.iacr.org/2018/1188">[URL]</a> <a href="#fnref:BBF18" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:BBF18:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a></p> </li> <li id="fn:SSY01" role="doc-endnote"> <p><strong>Blind, Auditable Membership Proofs</strong>, by Sander, Tomas and Ta-Shma, Amnon and Yung, Moti, <em>in Financial Cryptography</em>, 2001 <a href="#fnref:SSY01" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:TXN20e" role="doc-endnote"> <p><strong>Authenticated Dictionaries with Cross-Incremental Proof (Dis)aggregation</strong>, by Alin Tomescu and Yu Xia and Zachary Newman, <em>in Cryptology ePrint Archive, Report 2020/1239</em>, 2020, <a href="https://eprint.iacr.org/2020/1239">[URL]</a> <a href="#fnref:TXN20e" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Tome20" role="doc-endnote"> <p><strong>How to Keep a Secret and Share a Public Key (Using Polynomial Commitments)</strong>, by Tomescu, Alin, 2020, <a href="https://alinush.github.io/papers/phd-thesis-mit2020.pdf">[URL]</a> <a href="#fnref:Tome20" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:Tome20:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a></p> </li> </ol> </div>Alin TomescuAn RSA accumulator is an authenticated set built from cryptographic assumptions in hidden-order groups such as $\mathbb{Z}_N^*$. RSA accumulators enable a prover, who stores the full set, to convince any verifier, who only stores a succinct digest of the set, of various set relations such as (non)membership, subset or disjointness. For example, the prover can prove membership of elements in the set to verifiers who have the digest.Linear Diophantine Equations2020-11-23T00:00:00+00:002020-11-23T00:00:00+00:00https://alinush.github.io//2020/11/23/Linear-Diophantine-Equations<p>Equations of the form $\sum_i a_i x_i = 0$ where the $x_i$’s are <em>integer</em> unknowns are called <strong>linear Diophantine equations.</strong> Their integer solutions can be computed using greatest common denominator (GCD) tricks. In this post, we go over a few basic types of such equations and their integer solutions. <!--more--></p> <p hidden="">$$\def\lcm{\ \text{lcm}}$$</p> <h2 id="axby--t">$ax+by = t$</h2> <details> <summary><b>Theorem:</b> This equation has an integer solution $\Leftrightarrow \gcd(a,b) \mathrel\vert t$. </summary> <p style="margin-left: .3em; border-left: .15em solid black; padding-left: .5em;"> <em>Proof</em> (“$\Rightarrow$”): Assume an integer solution $(x_0, y_0)$ exists when $\gcd(a,b) \nmid t$. Since $\gcd(a,b)$ divides both $a$ and $b$, it divides any linear combination of them, including $ax_0 + by_0 = t$, which implies it divides $t$. Contradiction. <br /><br /> <em>Proof</em> (“$\Rightarrow$”): If $\gcd(a,b) = 1$, then an integer solution $x_0,y_0$ can be obtained using the Extended Eucliden algorithm, which finds $(u,v)$ such that $au + bv = \gcd(a,b)$. The solution is: \begin{align} x_0 &amp;= u\cdot t / \gcd(a,b)\\\ y_0 &amp;= v\cdot t / \gcd(a,b) \end{align} This is because $ax_0 + ay_0 = (au + bv) \frac{t}{\gcd(a,b)} = \gcd(a,b) \frac{t}{\gcd(a,b)} = t$. </p> </details> <details> <summary><b>Theorem:</b> When $\gcd(a,b) \mathrel\vert t$, if $(x_0, y_0)$ is an integer solution to such an equation, then all integer solutions can be characterized as: \begin{align} x &amp;= x_0 - k b/\gcd(a,b)\\\ y &amp;= y_0 + k a/\gcd(a,b) \end{align} Here, $k$ is an arbitrary integer and either $a\ne 0$ or $b\ne 0$ (since otherwise $\gcd(a,b) = 0$ and $\gcd(a,b)\mathrel\vert t$ implies $t=0$, which means all integers $(x,y)$ are solutions). </summary> <p style="margin-left: .3em; border-left: .15em solid black; padding-left: .5em;"> <em>Proof:</em> First, one can easily verify that the proposed $(x,y)$ are indeed solutions that satisfy $ax+by=t$: \begin{align} ax + by &amp;= ax_0 + kab / \gcd(a,b) + by_0 - kab/gcd(a,b)\\ &amp;= ax_0 + by_0 = t \end{align} The more difficult part is to argue that every solution has this form! Assume $a\ne 0$ since the other $b\ne 0$ case is symmetric. Assume $(x,y)$ to be an integer solution and note that, since $(x_0, y_0)$ is a solution, this means: \begin{align} a(x-x_0) + b(y - y_0) &amp;= 0\Leftrightarrow\\ a(x - x_0) &amp;= b(y_0 - y)\Leftrightarrow\\ \frac{a}{\gcd(a,b)}(x-x_0) &amp;= \frac{b}{\gcd(a,b)} (y_0 - y) \end{align} This implies $\frac{a}{\gcd(a,b)} \mathrel\vert \frac{b}{\gcd(a,b)} (y - y_0)$. Since $\gcd(\frac{a}{\gcd(a,b)}, \frac{b}{\gcd(a,b)}) = 1$<sup id="fnref:Heff03" role="doc-noteref"><a href="#fn:Heff03" class="footnote" rel="footnote">1</a></sup>, this means $\frac{a}{gcd(a,b)} \mathrel\vert (y-y_0)$, which means $\exists k$ such that: \begin{align} a/\gcd(a,b) \cdot k &amp;= (y-y_0)\Leftrightarrow\\ y &amp;= y_0 + k a / \gcd(a,b) \end{align} Next, substitute $y$ in $a(x-x_0) = b(y_0 - y)$, to get: \begin{align} a(x-x_0) &amp;= b(y_0 - y_0 - k a / \gcd(a,b))\Leftrightarrow\\ x - x_0 &amp;= (- b k a / \gcd(a,b)) / a\\ &amp;= x_0 - k b / \gcd(a,b) \end{align} Note that since $a\ne 0$, we are allowed to divide by $a$ above. </p> </details> <h3 id="axby--0">$ax+by = 0$</h3> <p>In this case, since $x_0 = 0$ and $y_0 = 0$ is one integer solution, all integer solutions are of the form: \begin{align} x &amp;= -kb/\gcd(a,b)\\<br /> y &amp;= ka/\gcd(a,b) \end{align}</p> <h2 id="axbycz--t">$ax+by+cz = t$</h2> <details> <summary><b>Theorem:</b> This equation has an integer solution $\Leftrightarrow \gcd(a,b,c) \mathrel\vert t$. </summary> <p style="margin-left: .3em; border-left: .15em solid black; padding-left: .5em;"> <em>Proof</em> (“$\Rightarrow$”): Proceeds analogously to the $ax+by = t$ case <a href="#axby--t">before</a>. <br /><br /> <em>Proof</em> (“$\Leftarrow$”): We will generalize the proof from the $ax+by = t$ case <a href="#axby--t">before</a>. <br /><br /> $\gcd(a,b,c) \mathrel\vert t \Rightarrow \gcd(a, \gcd(b, c)) \mathrel\vert t \Rightarrow ax + \gcd(b,c)w = t$ has an integer solution $(x_0, w_0)$. <br /><br /> Let $\gcd(b,c) = d$. We know $\exists (y_0, z_0)$ such that: \begin{align} b y_0 + c z_0 = d \end{align} Replacing in the previous equation, we have: \begin{align} ax_0 + \gcd(b,c)w_0 &amp;= t\Leftrightarrow\\ ax_0 + dw_0 &amp;= t\Leftrightarrow\\ ax_0 + (b y_0 + c z_0) w_0 &amp;= t\Leftrightarrow\\ ax_0 + b w_0 y_0 + c w_0 z_0 &amp;= t \end{align} Thus, an integer solution is $(x_0, y_0 w_0, z_0 w_0)$. </p> </details> <p><strong>Theorem</strong> <em>(from Sec 6.2 in <sup id="fnref:Cohe07" role="doc-noteref"><a href="#fn:Cohe07" class="footnote" rel="footnote">2</a></sup>)</em>: In this case, if $(x_0, y_0, z_0)$ is one integer solution, then all integer solutions are of the form: \begin{align} x &amp;= x_0 + m b / \gcd(a,b) - \ell c / \gcd(a,c)\\<br /> y &amp;= y_0 + k c / \gcd(b,c) - m a / \gcd(a, b)\\<br /> z &amp;= z_0 + \ell a / \gcd(a, c) - k b / \gcd(b, c) \end{align} Here, $m,\ell,k$ are integers and at least one of $a,b$ or $c$ are $\ne 0$.</p> <!-- _Proof:_ It is easy to check that proposed integers $x,y,z$ are indeed valid solutions. The more difficult part is to argue that every $(x,y,z)$ solution has this form! Assume $a\ne 0$ since the other $b\ne 0$ case is symmetric. Assume $(x,y,z)$ to be an integer solution and note that, since $(x_0, y_0, z_0)$ is a solution, this means: \begin{align} a(x-x_0) + b(y - y_0) + c(z-z_0) &= 0\Leftrightarrow\\\\\ a(x - x_0) &= b(y_0 - y) + c(z_0 - z)\Leftrightarrow\\\\\ \end{align} **TODO: continue** --> <h3 id="axbycz--0">$ax+by+cz = 0$</h3> <p>In this case, all integer solutions are of the form: \begin{align} x &amp;= m b / \gcd(a,b) - \ell c / \gcd(a,c)\\<br /> y &amp;= k c / \gcd(b,c) - m a / \gcd(a, b)\\<br /> z &amp;= \ell a / \gcd(a, c) - k b / \gcd(b, c) \end{align} Here, $m,\ell,k$ are arbitrary integers.</p> <!-- TODO: proof See page 10 here: https://people.math.sc.edu/howard/Classes/580f/hw5.pdf See https://math.stackexchange.com/questions/3325185/find-all-the-integral-solutions-to-the-equation-323x391y437z-10473/3327153#3327153 --> <p>This could also be simplified in terms of the <em>lowest common multiple (LCM)</em>, since $a b = \gcd(a,b) \lcm(a,b)\Rightarrow b / \gcd(a,b) = \lcm(a,b) / a$: \begin{align} x &amp;= m \lcm(a,b) / a - \ell \lcm(a,c) / a = \frac{m\lcm(a,b) - \ell\lcm(a,c)}{a}\\<br /> y &amp;= k \lcm(b,c) / b - m \lcm(a, b) / b = \frac{k\lcm(b,c) - m\lcm(a,b)}{b}\\<br /> z &amp;= \ell \lcm(a, c) / c - k \lcm(b, c) / c = \frac{\ell\lcm(a,c) - k\lcm(b,c)}{c} \end{align}</p> <h4 id="an-example">An example</h4> <p>In a Catalano-Fiore vector commitment (VC)<sup id="fnref:CF13e" role="doc-noteref"><a href="#fn:CF13e" class="footnote" rel="footnote">3</a></sup> of size $n=3$, collision resistance is implied by the fact that the following equation with $(\ell+1)$-bit primes $e_1,e_2,e_3$ does not have any $\ell$-bit integer solutions:</p> $e_2 e_3 v_1 + e_1 e_3 v_2 + e_1 e_2 v_3 = 0$ <p>The only integer solutions given by the formula above are at least $\ell+1$ bit wide: \begin{align} %x &amp;= \frac{m \lcm(e_2 e_3, e_1 e_3) - \ell\lcm(e_2 e_3, e_1 e_2)}{e_2 e_3}\\<br /> % &amp;= \frac{m e_1 e_2 e_3 - \ell e_1 e_2 e_3}{e_2 e_3} = (m-\ell)e_1\\<br /> x &amp;= m (e_1 e_3) / \gcd(e_2 e_3, e_1 e_3) - \ell (e_1 e_2) / \gcd(e_2 e_3, e_1 e_2) =\\<br /> &amp;= m (e_1 e_3) / e_3 - \ell (e_1 e_2) / e_2 = m’ e_1\ (\text{where}\ m’=m-\ell)\\<br /> y &amp;= k’ e_2\\<br /> z &amp;= \ell’ e_3 \end{align}</p> <h2 id="sum_iinn-a_i-x_i--t">$\sum_{i\in[n]} a_i x_i = t$</h2> <p><strong>Theorem:</strong> This equation has an integer solution $\Leftrightarrow \gcd(a_1, \dots, a_n) \mathrel\vert t$.</p> <p><em>Proof:</em> The “$\Rightarrow$” direction proceeds as <a href="#axbycz--t">before</a>.</p> <p>The “$\Leftarrow$” direction proceeds by induction. The statement $P(n)$ being proved by induction is:</p> $\forall a_i\in \mathbb{Z}, \gcd(a_1, \dots, a_n) \mathrel\vert t\Rightarrow \sum_{i\in[n]} a_i x_i = t\ \text{has an integer solution.}$ <p>We prove $P(n)$ is true for all $n \ge 2$. First, $P(2)$ is clearly true as we’ve shown <a href="#axby--t">before</a>. Second, we must show $P(n) \Rightarrow P(n + 1)$. This we do analogously to the proof for $n=3$ from <a href="#axbycz--t">before</a>. Let $d = \gcd(a_1, \dots, a_n)$. We know that $\gcd(a_1, \dots, a_{n+1}) = \gcd(d, a_{n+1})$. Since $d \mathrel| t$ (because $P(n)$ is true), this implies $\exists$ integers $s_0, w_0$ such that: \begin{align} d s_0 + a_{n+1} w_0 = t \Leftrightarrow \end{align}</p> <p>Since $P(n)$ is true, we know there exist integers $s_1,\dots, s_n$ such that: \begin{align} \sum_{i\in[n]} a_i s_i = d \end{align}</p> <p>Replacing $d$ in the previous equation, we get: \begin{align} d s_0 + a_{n+1} w_0 &amp;= t \Leftrightarrow\\<br /> \left(\sum_{i\in[n]} a_i s_i\right) s_0 + a_{n+1} w_0 &amp;= t \end{align} Thus, a solution can be found by setting $x_i = s_0 s_i,\forall i\in[n]$ and $x_{n+1} = w_0$.</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:Heff03" role="doc-endnote"> <p><strong>Elementary Number Theory</strong>, by Jim Hefferson and W. Edwin Clark, 2004, <a href="http://joshua.smcvt.edu/numbertheory/book.pdf">[URL]</a> <a href="#fnref:Heff03" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Cohe07" role="doc-endnote"> <p><strong>Number Theory: Volume I: Tools and Diophantine Equations</strong>, by Cohen, H., 2007, <a href="https://books.google.com/books?id=8zC8VPQV8psC">[URL]</a> <a href="#fnref:Cohe07" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:CF13e" role="doc-endnote"> <p><strong>Vector Commitments and their Applications</strong>, by Dario Catalano and Dario Fiore, <em>in Cryptology ePrint Archive, Report 2011/495</em>, 2011, <a href="https://eprint.iacr.org/2011/495">[URL]</a> <a href="#fnref:CF13e" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div>Alin TomescuEquations of the form $\sum_i a_i x_i = 0$ where the $x_i$’s are integer unknowns are called linear Diophantine equations. Their integer solutions can be computed using greatest common denominator (GCD) tricks. In this post, we go over a few basic types of such equations and their integer solutions.Cryptographic Assumptions in Hidden-Order Groups2020-11-05T00:00:00+00:002020-11-05T00:00:00+00:00https://alinush.github.io//2020/11/05/cryptographic-assumptions-in-hidden-order-groups<!-- **TODO:** - Define $\QRn$ + Include Quadratic Residuosity Assumption (QRA) (see [here](https://en.wikipedia.org/wiki/Quadratic_residuosity_problem) and [here](https://crypto.stanford.edu/pbc/notes/numbertheory/qr.html)) - Define **generic group model (GGM)** and **generic adversary**[^DK02] - _Blum integers_ as defined by Hastad et al.[^HSS93] - One-more RSA assumptions --> <p>In this post, we summarize some of the cryptographic hardness assumptions used in <strong>hidden-order groups</strong>.</p> <!--more--> <p hidden="">$$\def\Adv{\mathcal{A}} \def\Badv{\mathcal{B}} \def\GenGho{\mathsf{GenGroup}_?} \def\Ghosz{|\Gho|} \def\Ghoid{1_{\Gho}} \def\primes#1{\mathsf{Primes}_{#1}} \def\QRn{\mathsf{QR}_N}$$</p> <h2 id="terminology-and-notation">Terminology and notation</h2> <p>We try to describe these assumptions in terms of a generic hidden-order group $\Gho$ of order $\Ghosz$. We denote the identity element in such a group by $\Ghoid$.</p> <p>Sometimes, we specifically refer to the RSA group $\Gho=\ZNs$. Specifically, let $N = pq$ be the product of two sufficiently-large prime integers $p, q$. Then, $$\ZNs = \{0 &lt; a &lt; N \mathrel| \gcd(a, N) = 1\}$$ is the multiplicative group of integers co-prime with $N$. Recall that the size or <em>order</em> of this group is given by the totient function: $|\ZNs| = \phi(N) = (p-1)(q-1)$.</p> <p>Other times we might also refer to the <em>class group of imaginary quadratic orders</em> introduced by Buchmann and Williams<sup id="fnref:BW88" role="doc-noteref"><a href="#fn:BW88" class="footnote" rel="footnote">1</a></sup>.</p> <p class="warning">When we say <em>“Assumption $A$ implies assumption $B$”</em> this means that <em>“for $A$ to hold, $B$ must hold”</em>. Or, put differently, this means that <em>“$A$ can be broken given an algorithm that breaks $B$”</em>. For example, to say that <em>“the Strong RSA assumption implies the order assumption (OA)”</em> is the same thing as saying <em>“the Strong RSA problem reduces to the order problem”</em>, which is the same thing as saying <em>“given an algorithm for solving the order problem, one can solve the Strong RSA problem.”</em></p> <p>All of our probability clauses $\Pr[\dots]=\negl(\lambda)$ below would more formally be stated as “for all <em>polynomial probabilistic time (PPT)</em> adversaries $\Adv$, there exists a negligible function $\negl(\cdot)$ such that $\Pr[\dots] = \negl(\lambda)$.</p> <h2 id="assumptions">Assumptions</h2> <h3 id="factoring">Factoring</h3> $\Pr \begin{bmatrix} p, q \xleftarrow{\}\primes{\poly(\lambda)},\\ N = pq : \\ (p,q) \leftarrow \Adv(N) \\ \end{bmatrix} \leq \negl(\lambda)$ <!-- **TODO:** - Eh, I'm actually not sure about how to precisely formalize the length of $p$ and $q$. - Factoring $n$ knowing $e$ and $d$[^Bone99] in more detail via [StackExchange explantion](https://crypto.stackexchange.com/questions/78330/factoring-n-in-rsa-knowing-e-and-d) and [other SE answer](https://crypto.stackexchange.com/a/43228/21296) + Implementation in [cryptopp](https://github.com/weidai11/cryptopp/blob/CRYPTOPP_8_2_0/rsa.cpp#L160) --> <h3 id="discrete-logarithm">Discrete logarithm</h3> $\Pr \begin{bmatrix} \Gho \leftarrow \GenGho(\lambda),\\ (g,h) \xleftarrow{\} \Gho \times \Gho,\\ \ell \leftarrow \Adv(\Gho, g, h) : \\ g^{\ell} = h \end{bmatrix} \leq \negl(\lambda)$ <p><em>Bach</em><sup id="fnref:Bach84" role="doc-noteref"><a href="#fn:Bach84" class="footnote" rel="footnote">2</a></sup> shows that factoring $N$ reduces to computing discrete logs in $\ZNs$.</p> <!-- **TODO:** - Is this a trivial reduction? - What's the difference between restating the assumption as picking $\ell$ randomly and computing $h=g^{\ell}$. Most likely no difference except we'd have to ensure $\ell=O(\poly(\lambda))$ --> <h3 id="the-rsa-assumption-rsa">The RSA assumption (RSA)</h3> <p>Introduced by Rivest, Shamir and Adleman in their seminal 1978 paper on public-key cryptosystems<sup id="fnref:RSA78" role="doc-noteref"><a href="#fn:RSA78" class="footnote" rel="footnote">3</a></sup>.</p> <!-- **TODO:** Eh, need to bound the size of $\ell$ to be at most $\poly(\lambda)$. But is there a different class of assumptions based on such restrictions? --> $\Pr \begin{bmatrix} \Gho \leftarrow \GenGho(\lambda),\\ (g, \ell) \xleftarrow{\} \Gho \times \Z\ \text{s.t.}\ \gcd(\ell, \Ghosz)=1\\ u \leftarrow \Adv(\Gho, g, \ell) : \\ g^{1/\ell} = u \end{bmatrix} \leq \negl(\lambda)$ <!-- **TODO:** - Must $\ell$ be prime or not? BBHM02 assumes prime. - Do we need to enforce the $\gcd$ check or does picking $\ell$ randomly suffice? --> <p class="warning">When $\Gho=\ZNs$, we cannot have $\ell=2$ because it would not be co-prime with $\phi(N) = (p-1)(q-1)$ and $f(x) = x^\ell = x^2$ would not be a permutation. But if the subgroup of quadratic residues $\QRn$ is used, then $\ell = 2$ can be used (see <a href="https://crypto.stackexchange.com/a/65986/21296">here</a>). So it is best not to restrict the definition above.</p> <h3 id="strong-rsa-assumption">Strong RSA assumption</h3> <p>This assumption says that, given a random $g\in \Gho$, it is hard to find <em>any root</em> of it: i.e., an integer $\ell$ in $\Z$ and a group element $u$ such that $g^{1/\ell} = u$. Note that this generalizes the RSA assumption, which gives the adversary not only $g$, but also the root $\ell$ that should be computed.</p> $\Pr \begin{bmatrix} \Gho \leftarrow \GenGho(\lambda),\\ g\xleftarrow{\} \Gho,\\ (u,\ell) \leftarrow \Adv(\Gho, g) : \\ g^{1/\ell} = u\ \text{and}\ \ell &gt; 1 \end{bmatrix} \leq \negl(\lambda)$ <p><strong>Strong RSA assumption implies RSA assumption:</strong> If RSA assumption doesn’t hold then Strong RSA doesn’t either. To see this, consider a Strong RSA adversary $\Adv$ that gets a random $g$ as input. Since RSA doesn’t hold, there exists an RSA adversary $\Badv$ that given such a random $g$ and random $\ell\in \Z$ with $\gcd(\ell, \Ghosz) = 1$, outputs $u$ such that $g^{1/\ell} = u$. Note that $\Adv$ can use $\Badv$ to break Strong RSA! Specifically, $\Adv$ simply picks a random $\ell$ which, with overwhelming probability is co-prime with $\Ghosz$ (or else $\Adv$ can factor $N$ <!--**TODO: cite**-->). Next, $\Adv$ calls $\Badv$ with $g$ and $\ell$, obtaining $u$ such that $g^{1/\ell} = u$ with non-negligible probability. Finally, $\Adv$ simply outputs $(u,\ell)$, breaking Strong RSA with non-negligible probability.</p> <!-- **TODO:** - Again, what restrictions apply to $\ell$? Seems like $\ell > 2$ in RSA groups (though not sure how I would compute $u^{1/2}$ for random $u$. - I believe [LM18] considers composite $\ell$ as a different assumption. + Yes, and they show Strong RSA implies their assumption. --> <h3 id="the-order-assumption-oa">The order assumption (OA)</h3> <p>This assumption says that, given a random $g\in \Gho$, it is hard to find any multiple of its order: i.e., an integer $\ell$ such that $g^\ell = \Ghoid$ This is known as the <strong>order problem.</strong></p> <p class="warning">The earliest reference I could find to the order problem is in a paper by <em>Gary L. Miller</em><sup id="fnref:Mill76" role="doc-noteref"><a href="#fn:Mill76" class="footnote" rel="footnote">4</a></sup>, which shows it to be equivalent to factoring (under the Extended Riemann Hypothesis).</p> $\Pr \begin{bmatrix} \Gho \leftarrow \GenGho(\lambda),\\ g\xleftarrow{\} \Gho,\\ \ell \leftarrow \Adv(\Gho, g) : \\ g^\ell = \Ghoid \end{bmatrix} \leq \negl(\lambda)$ <p><em>Biehl et al.</em> highlight that <em>“The order problem can only be difficult if the order of random elements in G is large with a very high probability.”</em><sup id="fnref:BBHM02" role="doc-noteref"><a href="#fn:BBHM02" class="footnote" rel="footnote">5</a></sup></p> <p><strong>DL assumption implies OA:</strong> <em>Biehl et al.</em><sup id="fnref:BBHM02:1" role="doc-noteref"><a href="#fn:BBHM02" class="footnote" rel="footnote">5</a></sup> also give a reduction from the order problem to the discrete logarithm (DL) problem. Specifically, given an order problem instance $g$, if one can compute the DL $x$ of $g$ relative to $g^{-1}$ such that $g^x = g^{-1}$, then this gives a solution to the order problem as $g^{x+1} = \Ghoid$.</p> <p><strong>Strong RSA assumption implies OA:</strong> This is shown by Boneh et al.<sup id="fnref:BBF18" role="doc-noteref"><a href="#fn:BBF18" class="footnote" rel="footnote">6</a></sup> in Lemma 3. Suppose there is a <strong>generic</strong> adversary $\Adv$ who, given a (random) order problem $g$, solves it by outputting an $\ell$ such that $g^\ell = \Ghoid$. Then, there exists an adversary $\Badv$ that uses $\Adv$ to solve a random Strong RSA problem $g$. Specifically, $\Badv$ uses $\Adv$ to break the order problem $g$ and find the $\ell$ mentioned above. Then, $\Badv$ picks an odd prime $e$ that does not divide $\ell$ and outputs $u = g^{e^{-1} \bmod \ell}$ as the Strong RSA solution. Note that $e^{-1} \bmod \ell$ is notation for the multiplicative inverse of $e$ modulo $\ell$ (i.e., $e e^{-1} \equiv 1 \pmod \ell$). This inverse exists since $\gcd(e, \ell)=1$ (because $e$ is prime and does not divide $\ell$). The Extended Euclidean Algorithm (EEA) can be used to obtain the inverse $a = e^{-1} \bmod \ell$ by finding $(a,b)$ such that: <!-- TODO: Is \bmod \ell the right notation to use here? Or should we use \pmod \ell? This is what confused me initially. --></p> $ae + b\ell = 1 \Rightarrow a = (1 - b\ell)/e$ <p>Also, note that that $u^e = g$ and so $(u, \ell)$ are a solution for the Strong RSA problem $g$:</p> $u^e = (g^{e^{-1} \bmod \ell})^e = (g^\frac{1-b\ell}{e})^e = g^{1-b\ell} = g/(g^{\ell})^b= g$ <p><strong>RSA assumption implies OA:</strong> <em>Bieh et al.</em><sup id="fnref:BBHM02:2" role="doc-noteref"><a href="#fn:BBHM02" class="footnote" rel="footnote">5</a></sup> give a reduction from the RSA problem to the order problem<sup id="fnref:reduction" role="doc-noteref"><a href="#fn:reduction" class="footnote" rel="footnote">7</a></sup>. Specifically, given an RSA instance $(g, e)$ such that $e$ does not divide $\Ghosz$, if one can compute a multiple $\ell$ of the order of $g$ such that $g^\ell = \Ghoid$, then one can break the RSA instance as follows. We have two cases.</p> <ul> <li><em>First case:</em> If $e$ does not divide $\ell$, then we can break the assumption just like we broke Strong RSA above: output $u = g^{e^{-1} \bmod \ell}$.</li> <li><em>Second case:</em> If $e$ divides $\ell$, then we can let $\ell’ = \ell / e$. Importantly, since $e$ does not divide the order of $\Gho$, then $\ell’$ is still a multiple of the order of $g$<sup id="fnref:ellprime" role="doc-noteref"><a href="#fn:ellprime" class="footnote" rel="footnote">8</a></sup>. Thus, we can output $u = g^{e^{-1} \bmod \ell’}$ as before.</li> </ul> <!-- **TODO:** Why do they specifically use class groups in BBHM02, since RSA \QRn subgroups should also work, no?--> <p><strong>OA implies factoring assumption:</strong> We have to show that order problem reduces to factoring. Well, if you have a factoring oracle, you can factor $N=pq$, compute $\phi(N)= (p-1)(q-1)$ and then factor $\phi(N)$ as well. Next, given an order problem $g$, you know that $g^{\phi(N)} = \Ghoid$. If only multiples of the order are desired, then the problem is solved. Otherwise, if the actual order is required, then we know the order has to be a divisor of $\phi(N)$, the order of $\ZNs$. Thus, one can repeatedly divide out the divisors of $\phi(N)$ from the exponent and check if the result is still $\Ghoid$. For details, see <a href="https://math.stackexchange.com/questions/1025578/is-there-a-better-way-of-finding-the-order-of-a-number-modulo-n">this post</a> or see Algorithm 4.79 in Chapter 4 of the Handbook of Applied Cryptography<sup id="fnref:MvV96Ch4Pubkey" role="doc-noteref"><a href="#fn:MvV96Ch4Pubkey" class="footnote" rel="footnote">9</a></sup>.</p> <p><strong>Factoring assumption implies OA:</strong> <em>Miller</em><sup id="fnref:Mill76:1" role="doc-noteref"><a href="#fn:Mill76" class="footnote" rel="footnote">4</a></sup> shows that one can factor $N$ if one can compute the order of random group elements in $\ZNs$ (see Theorem 4). <em>Shor</em><sup id="fnref:Shor97" role="doc-noteref"><a href="#fn:Shor97" class="footnote" rel="footnote">10</a></sup> explains this reduction succinctly (see Section 5). <!-- But the OA assumption asks for multiples of the order, not for the order itself, so the reduction might not have access to the order. However, the RSA paper[^RSA78] says Miller's result reduction actually works with any multiple of the order. (However, I'm not sure I fully see this in Miller's paper.) --></p> <p><strong>OA in class groups:</strong> <em>Hamdy and Moller</em><sup id="fnref:HM00" role="doc-noteref"><a href="#fn:HM00" class="footnote" rel="footnote">11</a></sup> explain how to solve the order problem in class groups whose class number is smooth.</p> <!-- Questions and answers: - Is $\ell$ the actual order or a multiple of the order? + Seems like both work, so assuming multiple for generality. - Is [CF13e] referring to this assumption in their security proof? + Yes. They cite [HSS93], but [Mill76] would've been the right reference AFAICT. - How does the _Hastad et al._[^HSS93] result, which proves most elements in $\ZNs$ are of high order, relate to this assumption? + Well, if a significant fraction weren't of high order, then the random $g$ whose order must be found would be easy to solve. --> <!-- **TODOs:** - Strong RSA holds implies OA holds, seems to generalize the rsa.cash statement that Strong RSA implies OA when N is a product of safe primes. Is it true though? Maybe they are saying OA doesn't hold outside of $N$ = product of safe primes, so neither does Strong RSA. --> <h3 id="adaptive-root-assumption-ara">Adaptive root assumption (ARA)</h3> <p>This assumption was introduced by Wesolowski<sup id="fnref:Weso19" role="doc-noteref"><a href="#fn:Weso19" class="footnote" rel="footnote">12</a></sup>, who initially called it the <em>“root finding game”</em>.</p> $\Pr \begin{bmatrix} \Gho \leftarrow \GenGho(\lambda),\\ (g, \mathsf{state}) \xleftarrow{\} \Adv_0(\Gho),\\ \ell \xleftarrow{\} \primes{\lambda}, \\ u \leftarrow \Adv_1(\mathsf{state}, g,\ell): \\ g^{1/\ell} = u\ \text{and}\ g\ne \Ghoid \end{bmatrix} \leq \negl(\lambda)$ <!-- **TODO:** - How does the length of the primes affect this assumption? $\lambda$ versus $2\lambda$ versus 3? --> <h3 id="the-low-order-assumption-loa">The low-order assumption (LOA)</h3> <p>This assumption was first used by Pietrzak<sup id="fnref:Piet18e" role="doc-noteref"><a href="#fn:Piet18e" class="footnote" rel="footnote">13</a></sup> to construct <em>verifiable delay functions (VDFs)</em> and later formalized by Boneh et al.<sup id="fnref:BBF18A" role="doc-noteref"><a href="#fn:BBF18A" class="footnote" rel="footnote">14</a></sup> Informally, the assumptions says that the <strong>low-order problem</strong> is “hard” to solve. Specifically, given a group $\Gho$ of hidden-order, it is hard to find $g\in \Gho$ and $\ell \ne 0$ such that $g^{\ell} = \Ghoid$. <!-- In other words, it is hard to find the order of any element in $\Gho$. --></p> $\Pr \begin{bmatrix} \Gho \leftarrow \GenGho(\lambda),\\ (g, \ell) \leftarrow \Adv(\Gho) : \\ g^\ell = \Ghoid, g \ne \Ghoid\ \text{and}\ \ell &lt; 2^{\poly(\lambda)} \end{bmatrix} \leq \negl(\lambda)$ <p class="warning"><strong>Distinction:</strong> LOA differs from the order assumption (OA) since the LOA adversary gets to choose both $g$ and $\ell$ such that $g^\ell = 1$. In contrast, in OA, the adversary was given a random $g$ and needed to find an $\ell$.</p> <!-- **TODO:** - it seems like the RSA paper mentions that any multiple of $\phi(N)$ can be used to factor $N$. + Does this immediately imply the low-order assumption must hold in these groups, because if you find $g, x$ such that $g^x = \Ghoid$ for a random group element, then $x$ must be a multiple of $\|\ZNs\| = \phi(N)$, no? Or it could be a multiple of any divisor of $\phi(N) = (p-1)(q-1)$ too I think? - How come low-order assumption doesn't imply factoring assumption? --> <p><strong>Adaptive root assumption implies LOA:</strong> Boneh et al.<sup id="fnref:BBF18A:1" role="doc-noteref"><a href="#fn:BBF18A" class="footnote" rel="footnote">14</a></sup> show that LOA holds in <em>generic groups</em><sup id="fnref:DK02" role="doc-noteref"><a href="#fn:DK02" class="footnote" rel="footnote">15</a></sup>, because breaking it implies breaking the adaptive root assumption, which holds generically (see Section 4).</p> <p><em>Seres and Burcsi</em><sup id="fnref:SB20" role="doc-noteref"><a href="#fn:SB20" class="footnote" rel="footnote">16</a></sup> explain that LOA is stronger than factoring arbitrary $N$ because in $\QRn$ all elements are of high order, so factoring $N$ will not help. However, for particular kinds of $N$, they show that factoring reduces to LOA.</p> <h2 id="other-notes">Other notes</h2> <p><em>Hohenberger</em><sup id="fnref:Hoh03" role="doc-noteref"><a href="#fn:Hoh03" class="footnote" rel="footnote">17</a></sup> introduces the notion of a <em>pseudo-free group</em> and <em>Rivest</em><sup id="fnref:Rive04" role="doc-noteref"><a href="#fn:Rive04" class="footnote" rel="footnote">18</a></sup> shows that if $\ZNs$ is pseudo-free, then $\ZNs$ satisfies Strong RSA and DL.</p> <p><em>Rabin</em><sup id="fnref:Rabi79" role="doc-noteref"><a href="#fn:Rabi79" class="footnote" rel="footnote">19</a></sup> shows that solving polynomial congruences $\phi(x) \equiv 0 \pmod N$ is as hard as factoring $N$ (see Section 5). He also shows that an algorithm, which given $(y,N)$, computes quadratic residues $x$ such that $x^2 = y \pmod N$ can be used to factor $N$ (see Theorem 1).</p> <p><em>Shor</em><sup id="fnref:Shor97:1" role="doc-noteref"><a href="#fn:Shor97" class="footnote" rel="footnote">10</a></sup> shows that finding the order $\ell$ of a random element $g$ in $\ZNs$ (i.e., $g^\ell = 1$) can be done using a <em>quantum computer</em> in polynomial time. (Shor specifically requires $\ell$ to be the <em>order</em>: i.e., the smallest such integer.) Then, Shor uses a reduction by <em>Miller</em><sup id="fnref:Mill76:2" role="doc-noteref"><a href="#fn:Mill76" class="footnote" rel="footnote">4</a></sup> to factor $N$, given the quantum oracle for solving the order problem. Stephanie Blanda gives an <a href="https://blogs.ams.org/mathgradblog/2014/04/30/shors-algorithm-breaking-rsa-encryption/">intuitive explanation</a>.</p> <p><em>Theorem:</em> For all $a &lt; N$, $\exists r\in \Z$, such that $a^r = 1 \pmod N$ iff. $\gcd(a, N)=1$ (see <a href="https://proofwiki.org/wiki/Integer_has_Multiplicative_Order_Modulo_n_iff_Coprime_to_n">here</a>). <!-- TODO: Eh, why did I write this down again? --></p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:BW88" role="doc-endnote"> <p><strong>A Key-Exchange System Based on Imaginary Quadratic Fields</strong>, by Johannes Buchmann and Hugh C. Williams, <em>in Journal of Cryptology</em>, 1988 <a href="#fnref:BW88" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Bach84" role="doc-endnote"> <p><strong>Discrete logarithms and factoring</strong>, by Eric Bach, 1984, <a href="https://www2.eecs.berkeley.edu/Pubs/TechRpts/1984/CSD-84-186.pdf">[URL]</a> <a href="#fnref:Bach84" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:RSA78" role="doc-endnote"> <p><strong>A method for obtaining digital signatures and public-key cryptosystems</strong>, by R. L. Rivest and A. Shamir and L. Adleman, <em>in Communications of the {ACM}</em>, 1978, <a href="https://doi.org/10.1145%2F359340.359342">[URL]</a> <a href="#fnref:RSA78" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Mill76" role="doc-endnote"> <p><strong>Riemann’s hypothesis and tests for primality</strong>, by Gary L. Miller, <em>in Journal of Computer and System Sciences</em>, 1976, <a href="http://www.sciencedirect.com/science/article/pii/S0022000076800438">[URL]</a> <a href="#fnref:Mill76" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:Mill76:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a> <a href="#fnref:Mill76:2" class="reversefootnote" role="doc-backlink">&#8617;<sup>3</sup></a></p> </li> <li id="fn:BBHM02" role="doc-endnote"> <p><strong>A Signature Scheme Based on the Intractability of Computing Roots</strong>, by Biehl, Ingrid and Buchmann, Johannes and Hamdy, Safuat and Meyer, Andreas, <em>in Designs, Codes and Cryptography</em>, 2002, <a href="https://doi.org/10.1023/A:1014927327846">[URL]</a> <a href="#fnref:BBHM02" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:BBHM02:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a> <a href="#fnref:BBHM02:2" class="reversefootnote" role="doc-backlink">&#8617;<sup>3</sup></a></p> </li> <li id="fn:BBF18" role="doc-endnote"> <p><strong>Batching Techniques for Accumulators with Applications to IOPs and Stateless Blockchains</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/1188</em>, 2018, <a href="https://eprint.iacr.org/2018/1188">[URL]</a> <a href="#fnref:BBF18" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:reduction" role="doc-endnote"> <p>A <em>reduction</em> from the RSA problem to the order problem is an algorithm that solves the RSA problem given an oracle for solving the order problem. <a href="#fnref:reduction" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:ellprime" role="doc-endnote"> <p>One way to see why $g^{\ell’} = \Ghoid$ is to note that $g^{\ell} = g^{\ell’ e} = \Ghoid$ and, since $e$ does not divide $\Ghosz$ and $e$ is prime, it follows that $\gcd(e, \Ghosz)$ = 1. Thus, $e$ can be inverted. Raising $g^{\ell’ e} = \Ghoid$ to the inverse $e^{-1}$ gives $g^{\ell’} = \Ghoid$. <a href="#fnref:ellprime" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:MvV96Ch4Pubkey" role="doc-endnote"> <p><strong>Public-Key Parameters</strong>, by Menezes, Alfred J and van Oorschot, Paul C and Vanstone, Scott A, <em>in Handbook of Applied Cryptography</em>, 1996, <a href="http://cacr.uwaterloo.ca/hac/about/chap4.pdf">[URL]</a> <a href="#fnref:MvV96Ch4Pubkey" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Shor97" role="doc-endnote"> <p><strong>Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer</strong>, by Shor, Peter W., <em>in SIAM Journal on Computing</em>, 1997, <a href="https://doi.org/10.1137/S0097539795293172">[URL]</a> <a href="#fnref:Shor97" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:Shor97:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a></p> </li> <li id="fn:HM00" role="doc-endnote"> <p><strong>Security of Cryptosystems Based on Class Groups of Imaginary Quadratic Orders</strong>, by Hamdy, Safuat and M"oller, Bodo, <em>in Advances in Cryptology — ASIACRYPT 2000</em>, 2000 <a href="#fnref:HM00" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Weso19" role="doc-endnote"> <p><strong>Efficient Verifiable Delay Functions</strong>, by Wesolowski, Benjamin, <em>in Advances in Cryptology – EUROCRYPT 2019</em>, 2019 <a href="#fnref:Weso19" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Piet18e" role="doc-endnote"> <p><strong>Simple Verifiable Delay Functions</strong>, by Krzysztof Pietrzak, <em>in Cryptology ePrint Archive, Report 2018/627</em>, 2018, <a href="https://eprint.iacr.org/2018/627">[URL]</a> <a href="#fnref:Piet18e" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:BBF18A" role="doc-endnote"> <p><strong>A Survey of Two Verifiable Delay Functions</strong>, by Dan Boneh and Benedikt Bünz and Ben Fisch, <em>in Cryptology ePrint Archive, Report 2018/712</em>, 2018, <a href="https://eprint.iacr.org/2018/712">[URL]</a> <a href="#fnref:BBF18A" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:BBF18A:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a></p> </li> <li id="fn:DK02" role="doc-endnote"> <p><strong>Generic Lower Bounds for Root Extraction and Signature Schemes in General Groups</strong>, by Damg\aard, Ivan and Koprowski, Maciej, <em>in Advances in Cryptology — EUROCRYPT 2002</em>, 2002 <a href="#fnref:DK02" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:SB20" role="doc-endnote"> <p><strong>A Note on Low Order Assumptions in RSA groups</strong>, by István András Seres and Péter Burcsi, <em>in Cryptology ePrint Archive, Report 2020/402</em>, 2020, <a href="https://eprint.iacr.org/2020/402">[URL]</a> <a href="#fnref:SB20" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Hoh03" role="doc-endnote"> <p><strong>The Cryptographic Impact of Groups with Infeasible Inversion</strong>, by Susan Rae Hohenberger, <em>Master’s Thesis, MIT</em>, 2003 <a href="#fnref:Hoh03" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Rive04" role="doc-endnote"> <p><strong>On the Notion of Pseudo-Free Groups</strong>, by Rivest, Ronald L., <em>in Theory of Cryptography</em>, 2004 <a href="#fnref:Rive04" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:Rabi79" role="doc-endnote"> <p><strong>Digitalized Signatures and Public-key Functions as Intractable as Factorization</strong>, by Rabin, M. O., 1979 <a href="#fnref:Rabi79" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div>Alin TomescuIn this post, we summarize some of the cryptographic hardness assumptions used in hidden-order groups.Kate-Zaverucha-Goldberg (KZG) Constant-Sized Polynomial Commitments2020-05-06T22:38:00+00:002020-05-06T22:38:00+00:00https://alinush.github.io//2020/05/06/kzg-polynomial-commitments<p>Kate, Zaverucha and Goldberg introduced a constant-sized polynomial commitment scheme in 2010<sup id="fnref:KZG10e" role="doc-noteref"><a href="#fn:KZG10e" class="footnote" rel="footnote">1</a></sup>. We refer to this scheme as <strong>KZG</strong> and quickly introduce it below.</p> <p><strong>Prerequisites:</strong></p> <ul> <li>Cyclic groups of prime order and finite fields $\Zp$</li> <li>Pairings (or bilinear maps)</li> <li><a href="/2020/03/16/polynomials-for-crypto.html">Polynomials</a></li> </ul> <h2 id="trusted-setup">Trusted setup</h2> <p>To commit to degree $\le \ell$ polynomials, need $\ell$-SDH public parameters: $$(g,g^\tau,g^{\tau^2},\dots,g^{\tau^\ell}) = (g^{\tau^i})_{i\in[0,\ell]}$$</p> <p>Here, $\tau$ is called the <strong>trapdoor</strong>. These parameters should be generated via a distributed protocol<sup id="fnref:BCGplus15" role="doc-noteref"><a href="#fn:BCGplus15" class="footnote" rel="footnote">2</a></sup>$^,$<sup id="fnref:BGG18" role="doc-noteref"><a href="#fn:BGG18" class="footnote" rel="footnote">3</a></sup>$^,$<sup id="fnref:BGM17" role="doc-noteref"><a href="#fn:BGM17" class="footnote" rel="footnote">4</a></sup> that outputs just the $g^{\tau^i}$’s and <strong>forgets the trapdoor</strong> $\tau$.</p> <p>The public parameters are <strong>updatable</strong>: given $g^{\tau^i}$’s, anyone can update them to $g^{\alpha^i}$’s where $\alpha = \tau + \Delta$ by picking a random $\Delta$ and computing: $$g^{\alpha^i} = \left(g^{\tau^i}\right)^{\Delta^i}$$</p> <p>This is useful when you want to safely re-use a pre-generated set of public parameters, without trusting that nobody knows the trapdoor.</p> <h2 id="commitments">Commitments</h2> <p>Commitment to $\phi(X)=\sum_{i\in[0,d]} \phi_i X^i$ is $c=g^{\phi(\tau)}$ computed as:</p> $c=\prod_{i\in[0,\deg{\phi}]} \left(g^{\tau^i}\right)^{\phi_i}$ <p>Since it is just one group element, the commitment is <em>constant-sized</em>.</p> <h2 id="evaluation-proofs">Evaluation proofs</h2> <p>To prove an evaluation $\phi(a) = y$, a <em>quotient polynomial</em> is computed in $O(d)$ time:</p> $q(X) = \frac{\phi(X) - y}{X - a}$ <p>Then, the <em>constant-sized</em> <strong>evaluation proof</strong> is:</p> $\pi = g^{q(\tau)}$ <p>Note that this leverages the <a href="/2020/03/16/polynomials-for-crypto.html#the-polynomial-remainder-theorem">polynomial remainder theorem</a>.</p> <h3 id="verifying-an-evaluation-proof">Verifying an evaluation proof</h3> <p>A verifier who has the commitment $c=g^{\phi(\tau)}$, the evaluation $y=\phi(a)$ and the proof $\pi=g^{q(\tau)}$ can verify the evaluation in <em>constant-time</em> using two pairings:</p> <p>\begin{align} e(c / g^y, g) &amp;= e(\pi, g^\tau / g^a) \Leftrightarrow\\<br /> e(g^{\phi(\tau)-y}, g) &amp;= e(g^{q(\tau)}, g^{\tau-a}) \Leftrightarrow\\<br /> e(g,g)^{\phi(\tau)-y} &amp;= e(g,g)^{q(\tau)(\tau-a)}\\<br /> \phi(\tau)-y &amp;= q(\tau)(\tau-a) \end{align}</p> <p>This effectively checks that $q(X) = \frac{\phi(X) - y}{X-a}$ by checking this equality holds for $X=\tau$. In other words, it checks that the <a href="/2020/03/16/polynomials-for-crypto.html#the-polynomial-remainder-theorem">polynomial remainder theorem</a> holds at $X=\tau$.</p> <h2 id="batch-proofs">Batch proofs</h2> <p>One can prove multiple evaluations $(\phi(e_i) = y_i)_{i\in I}$ for <em>arbitrary</em> points $e_i$ using a constant-sized <strong>KZG batch proof</strong> $\pi_I = g^{q_I(\tau)}$, where:</p> <p>\begin{align} \label{eq:batch-proof-rel} q_I(X) &amp;=\frac{\phi(X)-R_I(X)}{A_I(X)}\\<br /> A_I(X) &amp;=\prod_{i\in I} (X - e_i)\\<br /> R_I(e_i) &amp;= y_i,\forall i\in I\\<br /> \end{align}</p> <p>$R_I(X)$ can be interpolated via <a href="/2020/03/16/polynomials-for-crypto.html#lagrange-interpolation">Lagrange interpolation</a> in $O(\vert I\vert\log^2{\vert I\vert})$ time<sup id="fnref:vG13ModernCh10" role="doc-noteref"><a href="#fn:vG13ModernCh10" class="footnote" rel="footnote">5</a></sup> as:</p> <p>\begin{align} R_I(X)=\sum_{i\in I} y_i \prod_{j\in I,j\ne i}\frac{X - e_j}{e_i - e_j} \end{align}</p> <p>$A_I(X)$ can be computed in $O(\vert I \vert \log^2{\vert I \vert})$ time via a <strong>subproduct tree</strong> in $O(\vert I\vert\log^2{\vert I\vert})$ time<sup id="fnref:vG13ModernCh10:1" role="doc-noteref"><a href="#fn:vG13ModernCh10" class="footnote" rel="footnote">5</a></sup>, as depicted below (for $\vert I \vert = 8$). The computation proceeds downwards, in the direction of the arrows, with the $(X-e_i)$ monomials being computed first.</p> <p><img src="/pictures/accumulator-subproduct-tree-inverted.png" /></p> <p>Each node in the subproduct tree multiplies the polynomials stored in its two children nodes. This way, the root polynomial will be exactly $A_I(X)$. If FFT-based multiplication is used, the time to compute a subproduct tree of size $n$ is:</p> <p>\begin{align} T(n) &amp;= 2T(n/2) + O(n\log{n})\\<br /> &amp;= O(n\log^2{n}) \end{align}</p> <p class="info"><strong>Observation 1:</strong> I believe computing $A_I(X)$ faster for <em>arbitrary</em> points $e_i$ is not possible, but I would be happy to be contradicted!</p> <p class="info"><strong>Observation 2:</strong> In practice, the algorithms for computing $R_I(X)$ and $A_I(X)$ <em>efficiently</em> would require FFT-based techniques for polynomial division and multiplication, and FFTs are fastest when the finite field $\Zp$ is endowed with $d$th roots of unity for sufficiently high $d$, on the order of the degrees of $R_I(X)$ and $A_I(X)$.</p> <h3 id="verifying-a-batch-proof">Verifying a batch proof</h3> <p>The verifier who has the commitment $c$, the evaluations $(e_i, y_i)_{i\in I}$ and a batch proof $\pi_I=g^{q_I(\tau)}$ can verify them as follows.</p> <ol> <li>First, he interpolates the <strong>accumulator polynomial</strong> $$A_I(X)=\prod_{i\in I} (X-e_i)$$ as discussed above. Then, commits to in $O(\vert I \vert)$ time: \begin{align} a &amp;= g^{A_I(\tau)} \end{align}</li> <li>Second, he interpolates $R_I(X)$ s.t. $R_I(e_i)=y_i,\forall i \in I$ as discussed above. Then, commits to in $O(\vert I \vert)$ time: \begin{align} r &amp;= g^{R_I(\tau)} \end{align}</li> <li>Third, he checks Equation \ref{eq:batch-proof-rel} holds at $X=\tau$ using two pairings: $e(c / r, g) = e(\pi_I, a)$.</li> </ol> <p>Note that:</p> <p>\begin{align} e(g^{\phi(\tau)} / g^{R_I(\tau)}, g) &amp;= e(g^{q_I(\tau)}, g^{A_I(\tau)})\Leftrightarrow\\<br /> e(g^{\phi(\tau) - R_I(\tau)}, g) &amp;= e(g,g)^{q_I(\tau) A_I(\tau)}\Leftrightarrow\\<br /> \phi(\tau) - R_I(\tau) &amp;= q_I(\tau) A_I(\tau) \end{align}</p> <!-- TODO: ## Commitment and proof homomorphism --> <h2 id="aggregation-of-proofs">Aggregation of proofs</h2> <p>For now, we discuss proof aggregation in <a href="/2020/05/06/aggregatable-subvector-commitments-for-stateless-cryptocurrencies.html#aggregating-proofs-into-subvector-profs">a different blog post on building vector commitments (VCs)</a> from KZG.</p> <!-- TODO: proof updates --> <!-- TODO: ## Information-theoretic hiding --> <!-- TODO: ### security proofs --> <h2 id="applications">Applications</h2> <p>There are many cryptographic tools one can build using polynomial commitment schemes such as KZG.</p> <p>Here’s a few we’ve blogged about in the past:</p> <ul> <li><a href="/2020/04/02/bilinear-accumulators-for-cryptocurrency.html">Cryptographic accumulators</a></li> <li>Vector Commitments (VC) schemes <a href="/2020/03/12/towards-scalable-vss-and-dkg.html">with $O(\log{n})$-sized proofs</a> or <a href="/2020/05/06/aggregatable-subvector-commitments-for-stateless-cryptocurrencies.html">with $O(1)$-sized proofs</a></li> <li><a href="/2020/03/03/range-proofs-from-polynomial-commitments-reexplained.html">Range proofs</a></li> </ul> <h4 id="acknowledgements">Acknowledgements</h4> <p>Many thanks to <a href="http://sshravan.github.io/">Shravan Srinivasan</a> and <a href="https://philipp.jovanovic.io/">Philipp Jovanovic</a> for <em>really</em> helping improve this post.</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:KZG10e" role="doc-endnote"> <p><strong>Polynomial commitments</strong>, by Kate, Aniket and Zaverucha, Gregory M and Goldberg, Ian, 2010, <a href="https://pdfs.semanticscholar.org/31eb/add7a0109a584cfbf94b3afaa3c117c78c91.pdf">[URL]</a> <a href="#fnref:KZG10e" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:BCGplus15" role="doc-endnote"> <p><strong>Secure Sampling of Public Parameters for Succinct Zero Knowledge Proofs</strong>, by E. Ben-Sasson and A. Chiesa and M. Green and E. Tromer and M. Virza, <em>in 2015 IEEE Symposium on Security and Privacy</em>, 2015 <a href="#fnref:BCGplus15" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:BGG18" role="doc-endnote"> <p><strong>A Multi-party Protocol for Constructing the Public Parameters of the Pinocchio zk-SNARK</strong>, by Bowe, Sean and Gabizon, Ariel and Green, Matthew D., <em>in Financial Cryptography and Data Security</em>, 2019 <a href="#fnref:BGG18" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:BGM17" role="doc-endnote"> <p><strong>Scalable Multi-party Computation for zk-SNARK Parameters in the Random Beacon Model</strong>, by Sean Bowe and Ariel Gabizon and Ian Miers, 2017, <a href="https://eprint.iacr.org/2017/1050">[URL]</a> <a href="#fnref:BGM17" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:vG13ModernCh10" role="doc-endnote"> <p><strong>Fast polynomial evaluation and interpolation</strong>, by von zur Gathen, Joachim and Gerhard, Jurgen, <em>in Modern Computer Algebra</em>, 2013 <a href="#fnref:vG13ModernCh10" class="reversefootnote" role="doc-backlink">&#8617;</a> <a href="#fnref:vG13ModernCh10:1" class="reversefootnote" role="doc-backlink">&#8617;<sup>2</sup></a></p> </li> </ol> </div>Alin TomescuKate, Zaverucha and Goldberg introduced a constant-sized polynomial commitment scheme in 20101. We refer to this scheme as KZG and quickly introduce it below. Prerequisites: Cyclic groups of prime order and finite fields $\Zp$ Pairings (or bilinear maps) Polynomials Trusted setup To commit to degree $\le \ell$ polynomials, need $\ell$-SDH public parameters: $$(g,g^\tau,g^{\tau^2},\dots,g^{\tau^\ell}) = (g^{\tau^i})_{i\in[0,\ell]}$$ Here, $\tau$ is called the trapdoor. These parameters should be generated via a distributed protocol2$^,$3$^,$4 that outputs just the $g^{\tau^i}$’s and forgets the trapdoor $\tau$. The public parameters are updatable: given $g^{\tau^i}$’s, anyone can update them to $g^{\alpha^i}$’s where $\alpha = \tau + \Delta$ by picking a random $\Delta$ and computing: $$g^{\alpha^i} = \left(g^{\tau^i}\right)^{\Delta^i}$$ This is useful when you want to safely re-use a pre-generated set of public parameters, without trusting that nobody knows the trapdoor. Commitments Commitment to $\phi(X)=\sum_{i\in[0,d]} \phi_i X^i$ is $c=g^{\phi(\tau)}$ computed as: $c=\prod_{i\in[0,\deg{\phi}]} \left(g^{\tau^i}\right)^{\phi_i}$ Since it is just one group element, the commitment is constant-sized. Evaluation proofs To prove an evaluation $\phi(a) = y$, a quotient polynomial is computed in $O(d)$ time: $q(X) = \frac{\phi(X) - y}{X - a}$ Then, the constant-sized evaluation proof is: $\pi = g^{q(\tau)}$ Note that this leverages the polynomial remainder theorem. Verifying an evaluation proof A verifier who has the commitment $c=g^{\phi(\tau)}$, the evaluation $y=\phi(a)$ and the proof $\pi=g^{q(\tau)}$ can verify the evaluation in constant-time using two pairings: \begin{align} e(c / g^y, g) &amp;= e(\pi, g^\tau / g^a) \Leftrightarrow\\ e(g^{\phi(\tau)-y}, g) &amp;= e(g^{q(\tau)}, g^{\tau-a}) \Leftrightarrow\\ e(g,g)^{\phi(\tau)-y} &amp;= e(g,g)^{q(\tau)(\tau-a)}\\ \phi(\tau)-y &amp;= q(\tau)(\tau-a) \end{align} This effectively checks that $q(X) = \frac{\phi(X) - y}{X-a}$ by checking this equality holds for $X=\tau$. In other words, it checks that the polynomial remainder theorem holds at $X=\tau$. Batch proofs One can prove multiple evaluations $(\phi(e_i) = y_i)_{i\in I}$ for arbitrary points $e_i$ using a constant-sized KZG batch proof $\pi_I = g^{q_I(\tau)}$, where: \begin{align} \label{eq:batch-proof-rel} q_I(X) &amp;=\frac{\phi(X)-R_I(X)}{A_I(X)}\\ A_I(X) &amp;=\prod_{i\in I} (X - e_i)\\ R_I(e_i) &amp;= y_i,\forall i\in I\\ \end{align} $R_I(X)$ can be interpolated via Lagrange interpolation in $O(\vert I\vert\log^2{\vert I\vert})$ time5 as: \begin{align} R_I(X)=\sum_{i\in I} y_i \prod_{j\in I,j\ne i}\frac{X - e_j}{e_i - e_j} \end{align} $A_I(X)$ can be computed in $O(\vert I \vert \log^2{\vert I \vert})$ time via a subproduct tree in $O(\vert I\vert\log^2{\vert I\vert})$ time5, as depicted below (for $\vert I \vert = 8$). The computation proceeds downwards, in the direction of the arrows, with the $(X-e_i)$ monomials being computed first. Each node in the subproduct tree multiplies the polynomials stored in its two children nodes. This way, the root polynomial will be exactly $A_I(X)$. If FFT-based multiplication is used, the time to compute a subproduct tree of size $n$ is: \begin{align} T(n) &amp;= 2T(n/2) + O(n\log{n})\\ &amp;= O(n\log^2{n}) \end{align} Observation 1: I believe computing $A_I(X)$ faster for arbitrary points $e_i$ is not possible, but I would be happy to be contradicted! Observation 2: In practice, the algorithms for computing $R_I(X)$ and $A_I(X)$ efficiently would require FFT-based techniques for polynomial division and multiplication, and FFTs are fastest when the finite field $\Zp$ is endowed with $d$th roots of unity for sufficiently high $d$, on the order of the degrees of $R_I(X)$ and $A_I(X)$. Verifying a batch proof The verifier who has the commitment $c$, the evaluations $(e_i, y_i)_{i\in I}$ and a batch proof $\pi_I=g^{q_I(\tau)}$ can verify them as follows. First, he interpolates the accumulator polynomial $$A_I(X)=\prod_{i\in I} (X-e_i)$$ as discussed above. Then, commits to in $O(\vert I \vert)$ time: \begin{align} a &amp;= g^{A_I(\tau)} \end{align} Second, he interpolates $R_I(X)$ s.t. $R_I(e_i)=y_i,\forall i \in I$ as discussed above. Then, commits to in $O(\vert I \vert)$ time: \begin{align} r &amp;= g^{R_I(\tau)} \end{align} Third, he checks Equation \ref{eq:batch-proof-rel} holds at $X=\tau$ using two pairings: $e(c / r, g) = e(\pi_I, a)$. Note that: \begin{align} e(g^{\phi(\tau)} / g^{R_I(\tau)}, g) &amp;= e(g^{q_I(\tau)}, g^{A_I(\tau)})\Leftrightarrow\\ e(g^{\phi(\tau) - R_I(\tau)}, g) &amp;= e(g,g)^{q_I(\tau) A_I(\tau)}\Leftrightarrow\\ \phi(\tau) - R_I(\tau) &amp;= q_I(\tau) A_I(\tau) \end{align} Aggregation of proofs For now, we discuss proof aggregation in a different blog post on building vector commitments (VCs) from KZG. Applications There are many cryptographic tools one can build using polynomial commitment schemes such as KZG. Here’s a few we’ve blogged about in the past: Cryptographic accumulators Vector Commitments (VC) schemes with $O(\log{n})$-sized proofs or with $O(1)$-sized proofs Range proofs Acknowledgements Many thanks to Shravan Srinivasan and Philipp Jovanovic for really helping improve this post. Polynomial commitments, by Kate, Aniket and Zaverucha, Gregory M and Goldberg, Ian, 2010, [URL] &#8617; Secure Sampling of Public Parameters for Succinct Zero Knowledge Proofs, by E. Ben-Sasson and A. Chiesa and M. Green and E. Tromer and M. Virza, in 2015 IEEE Symposium on Security and Privacy, 2015 &#8617; A Multi-party Protocol for Constructing the Public Parameters of the Pinocchio zk-SNARK, by Bowe, Sean and Gabizon, Ariel and Green, Matthew D., in Financial Cryptography and Data Security, 2019 &#8617; Scalable Multi-party Computation for zk-SNARK Parameters in the Random Beacon Model, by Sean Bowe and Ariel Gabizon and Ian Miers, 2017, [URL] &#8617; Fast polynomial evaluation and interpolation, by von zur Gathen, Joachim and Gerhard, Jurgen, in Modern Computer Algebra, 2013 &#8617; &#8617;2