<?xml version="1.0" encoding="UTF-8" ?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" version="2.0"><channel><title>Matt Hudson | CrunchyData Blog</title>
<atom:link href="https://www.crunchydata.com/blog/author/matt-hudson/rss.xml" rel="self" type="application/rss+xml" />
<link>https://www.crunchydata.com/blog/author/matt-hudson</link>
<image><url>https://www.crunchydata.com/build/_assets/default.png-W4XGD4DB.webp</url>
<title>Matt Hudson | CrunchyData Blog</title>
<link>https://www.crunchydata.com/blog/author/matt-hudson</link>
<width>256</width>
<height>256</height></image>
<description>PostgreSQL experts from Crunchy Data share advice, performance tips, and guides on successfully running PostgreSQL and Kubernetes solutions</description>
<language>en-us</language>
<pubDate>Fri, 18 Nov 2022 10:00:00 EST</pubDate>
<dc:date>2022-11-18T15:00:00.000Z</dc:date>
<dc:language>en-us</dc:language>
<sy:updatePeriod>hourly</sy:updatePeriod>
<sy:updateFrequency>1</sy:updateFrequency>
<item><title><![CDATA[ Postgres Query Boost: Using ANY Instead of IN ]]></title>
<link>https://www.crunchydata.com/blog/postgres-query-boost-using-any-instead-of-in</link>
<description><![CDATA[ Let’s show some love for the ANY Operator! Read about how to use = ANY(array) in place of the IN(list) so you can send all your parameters securely with prepared statements. ]]></description>
<content:encoded><![CDATA[ <p>The <code>IN</code> operator has to be in the top-3 SQL-specific operators that people learn, the close competitors are <code>LIKE</code> and <code>BETWEEN</code>. Thus, <code>IN</code> feels familiar when crafting SQL: you can use it with nested SQL statements, or create your own list. But, it does have limitations — what if you wanted to send a list of unknown length? You can’t really use SQL placeholders without also modifying the SQL statement.<p>What if I told you there was a less ubiquitous operator that offered all of the power and less of the drawbacks. In this article we'll talk about using <code>= ANY(array)</code> in instead of <code>IN(list)</code> to improve code and query quality.<p>We don't give Java enough love in our tutorials, so these examples are written for JDK with the <a href=https://vertx.io/docs/vertx-pg-client/java/>Vert.x reactive Postgres</a> library. Don’t worry, you’ll be able to read it. We chose a direct Postgres client because ORMs typically default to <code>IN(list)</code>, and I’m making an argument they should convert to <code>= ANY(array)</code> where applicable.<h2 id=unforeseenn-consequences><a href=#unforeseenn-consequences>Unforeseenn Consequences</a></h2><p>It starts simply enough: I have a collection of items that I want to filter by <code>id</code>. Perhaps, this is a user-selectable form that permits users to multi-select values. The most common query looks like this:<pre><code class=language-pgsql>SELECT i.*
FROM item AS i
JOIN shipment AS s ON s.item_id=i.id
WHERE s.status &#60> 'shipped' -- Grab only Unshipped items
  AND i.id IN (1,2,3,4,5,6);
</code></pre><p>Seems perfectly reasonable. I'll just copy that query into our code and replace the literal list with a placeholder right?<h2 id=introduce-complexity><a href=#introduce-complexity>Introduce Complexity</a></h2><p>Back in the code window, I drop it into some code approximately like this.<pre><code class=language-java>// Prepare Tuple of IDs
Tuple t_list = Tuple.of(1, 2, 3, 4, 5, 6);

// Prepare a parameterized query
pool.preparedQuery("SELECT i.*\\n" +
                "FROM item AS i\\n" +
                "JOIN shipment AS s ON s.item_id=i.id\\n" +
                "WHERE s.status &#60> 'shipped'\\n" +
                "  AND i.id IN ($1)")
				// Attempt to run it
        .execute(t_list)
				// Log the IDs returned
        .onSuccess(rows -> rows.forEach(
					row -> logger.info("id: " + row.getInteger("id"))))
				// Failure: Log the complaint
        .onFailure(t -> logger.error("QUERY FAIL", t));
</code></pre><p>But this doesn’t work. It will throw an exception about the wrong number of parameters. The <code>$1</code> placeholder expects a single value, but it receives 6 values. Aha, that <code>Tuple</code> must be packed wrong for the <code>IN</code> parameter. Maybe sending an array is better.<pre><code class=language-java>// Prepare Tuple of array of IDs
Integer[] items = {1, 2, 3, 4, 5, 6};
Tuple t_list = Tuple.of(items);

// Prepare a parameterized query
pool.preparedQuery("SELECT i.*\\n" +
                "FROM item AS i\\n" +
                "JOIN shipment AS s ON s.item_id=i.id\\n" +
                "WHERE s.status &#60> 'shipped'\\n" +
                "  AND i.id IN ($1)")
				// Attempt to run it
        .execute(t_list)
				// Log the IDs returned
        .onSuccess(rows -> rows.forEach(
					row -> logger.info("id: " + row.getInteger("id"))))
				// Failure: Log the complaint
        .onFailure(t -> logger.error("QUERY FAIL", t));
</code></pre><p>This doesn’t work either. It throws a wrong type error. Because Postgres is strongly typed, when comparing <code>id</code> SQL expects a <code>Number</code>, but gets <code>Integer[]</code>.<h2 id=time-to-read-the-fine-manual><a href=#time-to-read-the-fine-manual>Time to Read the Fine Manual</a></h2><p>The exceptions coming back from these attempts are not getting better. Let's head over to the <a href=https://www.postgresql.org/docs/current/functions-comparisons.html>PostgreSQL 15 documentation</a> to see if anything leaps out, regarding this.<blockquote><p><strong>9.24.1 <code>IN</code></strong><p>The right-hand side is a parenthesized list of scalar expressions.</blockquote><p>How do I send a parenthesized list of scalars? Is that even possible?<p>Well, I tried sending an array and I tried sending multiple scalars, but none of those matched a parenthesized list of scalar expressions. As it turns out, you can't prepare a variable list of scalar expressions. If you mean to send 5 scalars, you must write <code>IN ($1, $2, $3, $4, $5)</code>. This operator is finicky.<h2 id=the-case-for-any-or-some><a href=#the-case-for-any-or-some>The case for <code>ANY</code> (or <code>SOME</code>)</a></h2><p>Fortunately the solution is easily found on the same PostgreSQL 15 documentation page if I continue scrolling.<blockquote><p><strong>9.24.3. <code>ANY</code>/<code>SOME</code> (array)</strong><pre><code class=language-txt>	expression operator ANY (array expression)
	expression operator SOME (array expression)
</code></pre><p>The right-hand side is a parenthesized expression, which must yield an array value.</blockquote><p>Possible match? From the definitions, <code>expression IN (...)</code> is equivalent to <code>expression = ANY (...)</code> but for the parameter type!<h2 id=endgame><a href=#endgame>Endgame</a></h2><p>Let's give it a shot.<pre><code class=language-java>// Prepare Tuple of array of IDs
Integer[] items = {1, 2, 3, 4, 5, 6};
Tuple t_list = Tuple.of(items);

// Prepare a parameterized query
// this time replacing IN with =ANY
pool.preparedQuery("SELECT i.*\\n" +
                "FROM item AS i\\n" +
                "JOIN shipment AS s ON s.item_id=i.id\\n" +
                "WHERE s.status &#60> 'shipped'\\n" +
                "  AND i.id = ANY($1)")
				// Attempt to run it
        .execute(t_list)
				// Log the IDs returned
        .onSuccess(rows -> rows.forEach(
					row -> logger.info("id: " + row.getInteger("id") + " name: " + row.getString("name"))))
				// Failure: Log the complaint
        .onFailure(t -> logger.error("QUERY FAIL", t));
</code></pre><p>Success! My array binds as a single parameter to <code>ANY</code>.<h2 id=conclusions><a href=#conclusions>Conclusions</a></h2><p>Given the constraints on <code>IN</code> and the need to send parameters securely, it makes sense use <code>= ANY</code> where you might otherwise use <code>IN</code>.<p>In addition to parameter binding, it's worth mentioning that <code>ANY</code> works with all boolean comparison operators:<pre><code class=language-pgsql>* foo LIKE ANY ('{"%bar", "%baz"}')
* foo ILIKE ANY ('{"%bar", "%baz"}')
* id &#60> ANY ('{1, 2, 3}')
</code></pre> ]]></content:encoded>
<category><![CDATA[ Production Postgres ]]></category>
<author><![CDATA[ Matt.Hudson@crunchydata.com (Matt Hudson) ]]></author>
<dc:creator><![CDATA[ Matt Hudson ]]></dc:creator>
<guid isPermalink="false">ed30980c10a48460dd014224df106bae96cfbc953a3227def0f2c02bd7f8677e</guid>
<pubDate>Fri, 18 Nov 2022 10:00:00 EST</pubDate>
<dc:date>2022-11-18T15:00:00.000Z</dc:date>
<atom:updated>2022-11-18T15:00:00.000Z</atom:updated></item></channel></rss>