Fixing the MySQL Error “Numeric value out of range: 167 Out of range value for column ‘value_id’ at row 1” in Magento 2

Are you encountering the frustrating MySQL error “Numeric value out of range: 167 Out of range value for column ‘value_id’ at row 1” when you try to run your Magento 2 store? If so, you’re not alone. This error, stemming from a limitation in MySQL InnoDB database tables with an auto-increment column defined as int, can halt database operations and disrupt your workflow.

But fear not, for in the realm of MySQL troubleshooting, every challenge presents an opportunity for resolution. In this blog post, we embark on a journey to uncover the root cause of this error and implement a solution that restores the functionality of your database tables.

The crux of the issue lies in the inherent limitations of the int data type, which imposes a maximum value that can be stored in the auto-increment column. Once this maximum value is reached, further insertions into the table become impossible, resulting in the dreaded “out of range” error.

But fret not, as we delve into the intricacies of MySQL database management, we’ll explore practical strategies to overcome this limitation and restore the seamless operation of your database tables. From altering column data types to optimizing table structures, we’ll equip you with the knowledge and tools needed to address this error head-on.

The Debugging Process

A few days ago I faced an issue in a Magento 2 project that was very interesting to work on. A product integration was not performing what it was supposed to, so the products that were integrated were not set to be visible in the storefront. For some reason, the product data for some given products were not updated automatically and the store owner was forced to work on them manually to get them to appear to their customers but, from time to time, the product was kind of “reset” to the initial stage.

I started the process by checking the logs in the server and I saw some exceptions being logged in as follows:

SQLSTATE[22003]: Numeric value out of range: 167 Out of range value for column 'value_id' at row 1

By checking the details and the backtrace of this exception, I was able to track down the process. It’s a customization done for the client that imports the products from a given ERP and, at a certain point, it calls the following class and method:

\Magento\Catalog\Model\ResourceModel\Product\Action::updateAttributes

This method simply updates attribute values for an entity list per store.

When this method runs, the following query is executed against the MySQL database:

INSERT INTO `catalog_product_entity_int` (`attribute_id`,`store_id`,`value`,`row_id`)
VALUES (?, ?, ?, ?), (?, ?, ?, ?), (?, ?, ?, ?), (?, ?, ?, ?) ... 
ON DUPLICATE KEY UPDATE 
    `attribute_id` = VALUES(`attribute_id`), 
    `store_id` = VALUES(`store_id`), 
    `value` = VALUES(`value`), 
    `row_id` = VALUES(`row_id`)

Note: the quantity of (?, ?, ?, ?) this query will have will depend on how many entities (products) are being updated at this time.

When this query is executed, I got the exception.

SQLSTATE[22003]: Numeric value out of range: 167 Out of range value for column 'value_id' at row 1, query was: INSERT  INTO `catalog_product_entity_int` (`attribute_id`,`store_id`,`value`,`row_id`) VALUES (?, ?, ?, ?), (?, ?, ?, ?), (?, ?, ?, ?), (?, ?, ?, ?) ON DUPLICATE KEY UPDATE `attribute_id` = VALUES(`attribute_id`), `store_id` = VALUES(`store_id`), `value` = VALUES(`value`), `row_id` = VALUES(`row_id`) [] []

Why was that happening for a by-design Magento 2 operation? I mean, there was no customization on this part of the code.

The Root Cause of the Error

Obviously, I needed to dig deeper into this problem to understand the nuances and why that was happening. I did some research on the web to understand the principles of the error Numeric value out of range. I found this GitHub issue which gave me more context about the problem and, thanks to this, I was able to wrap my head around the issue and apply a proper solution to that.

Basically, MySQL database InnoDB tables that have auto-increment integer fields have their limits. In Magento 2, the main EAV tables for the products are the following:

• catalog_product_entity_int
• catalog_product_entity_datetime
• catalog_product_entity_decimal
• catalog_product_entity_text
• catalog_product_entity_varchar

All of them have a similar structure:

MariaDB [magento]> describe catalog_product_entity_int;
+--------------+----------------------+------+-----+---------+----------------+
| Field        | Type                 | Null | Key | Default | Extra          |
+--------------+----------------------+------+-----+---------+----------------+
| value_id     | int(11)              | NO   | PRI | NULL    | auto_increment |
| attribute_id | smallint(5) unsigned | NO   | MUL | 0       |                |
| store_id     | smallint(5) unsigned | NO   | MUL | 0       |                |
| value        | int(11)              | YES  |     | NULL    |                |
| row_id       | int(10) unsigned     | NO   | MUL | NULL    |                |
+--------------+----------------------+------+-----+---------+----------------+
5 rows in set (0.000 sec)

The value_id field is a signed int type and is an auto-increment field. That means that anytime something is inserted on this table, this field will control the ID of the row and will always increase the number.

If we go to the MySQL documentation page about the Integer Types, we’ll have the following information:

As we can see, the integer type values may vary from -2147483648 to 2147483647, when signed, and from 0 to 4294967295 when unsigned.

In Magento 2, the value_id field on these tables is signed instead of unsigned, but please, don’t ask me why. If they were unsigned, that means that we could have up the value_id up to the number 4294967295 on these tables, however, since they are signed, the value is only allowed up to 2147483647.

In this particular case, during the investigation, I checked the latest ID on the catalog_product_entity_int table and I found out that the next ID would be greater than the maximum allowed one, 2147483647.

It may not appear in the picture, but the next auto incremental ID would be something like the number 2,147,483,787, and that’s why no new rows were saved on this table anymore. That was the root cause.

OK, but… How to fix it?

During this investigation, I was able to find 3 different solutions to this issue. Two of them, even though they also work pretty well, I may call temporary ones, because the chances are higher that you might face this problem over again after some time. The third one is the one I applied as the final fix in my case, but feed free to test out the other approaches and check if they resolve your issue permanently or not.

Solution #1

The first one is based on these steps to basically “reset” the auto-increment values of the tables affected by this error. You:

1. Create a backup from your entire database;
2. Create a temp table with the same structure of the EAV one you’re trying to fix;
3. You transfer/copy all of the data from the original table to the new temp one;
4. You reset the auto-increment value on the original table;
5. Then you insert all the the data from the temp table back to the original one, disregarding the original value_id, so the increment IDs will start from 1.

Even though this approach fixes the issue pretty well, you may run into possible problems when transferring data from one table to another, which makes this operation riskier than the other ones.

You might be thinking why this fixes the issue if you’re simply transferring the data around and you might face the same issue because you will end up with the same amount of data when you finish it.

Yes, that’s true, you’re supposed to have the same amount of data after completing this operation, however, the number of rows is way less than what the auto-increment ID says. In my case, for instance, I had around 70,000 rows only, which is not even near to the auto-increment ID generated by the table.

Solution #2

The second solution is to make the value_id field unsigned instead of signed. This approach would double the number of IDs you can have on the table, as seen before. Instead of having the limit of 2147483647 IDs, you would have it limited to 4294967295, which would solve the problem. However, depending on the case, it would only be a matter of time to reach to the limit again. However, this approach is less risky than Solution #1.

To make this change, you can use an existing module or create a new one that will depend on the Magento_Catalog module and add the following content to etc/db_schema.xml:

<?xml version="1.0"?>
<schema xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Setup/Declaration/Schema/etc/schema.xsd">
    <table name="catalog_product_entity_int">
        <column name="value_id" unsigned="true"/>
    </table>
</schema>

The magic here is the unsigned="true" parameter. After this change is applied, if the module is new, you need to enable it and run the setup:upgrade:

> bin/magento module:enable My_Module
> bin/magento setup:upgrade

Note: I don’t know the implications of applying this change to these tables. Since Magento declares these fields are signed ones, you may be aware that applying this change might bring some implications or not.

Solution #3

This solution is the most proper one because it will increase the auto-increment capabilities considerably. This solution is pretty similar to Solution #2, which means it’s a low-risk approach as well, in terms of data loss and management. We need to change the type of the value_id field from int to biging.

As in Solution #2, you’ll need to add the following content to the etc/db_schema.xml file on either an existing or a new module.

<?xml version="1.0"?>
<schema xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Setup/Declaration/Schema/etc/schema.xsd">
    <table name="catalog_product_entity_datetime">
        <column xsi:type="bigint" name="value_id"/>
    </table>
    <table name="catalog_product_entity_decimal">
        <column xsi:type="bigint" name="value_id"/>
    </table>
    <table name="catalog_product_entity_int">
        <column xsi:type="bigint" name="value_id"/>
    </table>
    <table name="catalog_product_entity_text">
        <column xsi:type="bigint" name="value_id"/>
    </table>
    <table name="catalog_product_entity_varchar">
        <column xsi:type="bigint" name="value_id"/>
    </table>
</schema>

After this change, enable the module and run the bin/magento setup:upgrade CLI command to wrap up this process.

After applying this change, you might have the table(s) updated.

MariaDB [magento]> describe catalog_product_entity_int;
+--------------+----------------------+------+-----+---------+----------------+
| Field        | Type                 | Null | Key | Default | Extra          |
+--------------+----------------------+------+-----+---------+----------------+
| value_id     | bigint(20)           | NO   | PRI | NULL    | auto_increment |
| attribute_id | smallint(5) unsigned | NO   | MUL | 0       |                |
| store_id     | smallint(5) unsigned | NO   | MUL | 0       |                |
| value        | int(11)              | YES  |     | NULL    |                |
| row_id       | int(10) unsigned     | NO   | MUL | NULL    |                |
+--------------+----------------------+------+-----+---------+----------------+
5 rows in set (0.010 sec)

Conclusion

As we draw to a close on our journey to resolve the MySQL error “Numeric value out of range: 167,” we stand empowered with newfound knowledge and practical solutions to overcome database challenges.

By understanding the limitations of the int data type and the implications it holds for auto-increment columns in MySQL InnoDB tables, we’ve peeled back the layers of complexity surrounding this error. Armed with this insight, we’ve explored a range of strategies to address the issue, from altering column data types to optimizing table structures.

With each challenge we face, we grow in resilience and expertise, forging a path towards greater database stability and reliability. While MySQL errors may pose temporary setbacks, they also present opportunities for growth and learning. By confronting these challenges head-on, we strengthen our understanding of database management and deepen our ability to navigate the complexities of MySQL.

As you continue your journey in MySQL troubleshooting, may the knowledge gained from this exploration serve as a guiding light, illuminating the path forward and empowering you to overcome any obstacles that lie ahead. Remember, with patience, persistence, and a willingness to explore, there’s no MySQL error too daunting to conquer.

Here’s to a future filled with resilient databases, seamless operations, and continued growth in MySQL mastery. Let’s forge ahead with confidence, knowing that we’re well-equipped to handle whatever challenges come our way.

-Tiago