Сборка SAM не может разрешить зависимости для onnxruntime

Моя проблема

Краткое содержание

Я пытаюсь реализовать YOLO на AWS Lambda, и при попытке создать уровень Lambda с помощью ONNX я получаю Error: PythonPipBuilder:ResolveDependencies - {onnxruntime==1.18.1(wheel)}.

Минимальный пример

Вот минимальный шаблон, воспроизводящий ошибку:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  LambdaLayerOnnx:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: !Sub ${AWS::StackName}-onnx
      ContentUri: lambda/layers/onnx
    Metadata:
      BuildMethod: python3.12

Файл requirements.txt выглядит так:

onnxruntime==1.18.1 ; python_version >= "3.10" and python_version < "4.0"

При запуске с опцией --debug это логи важной части (отформатированные для удобства чтения):

2024-06-30 23:47:04,699 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)' 
2024-06-30 23:47:04,701 | Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)'
2024-06-30 23:47:04,703 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2024-06-30 23:47:04,785 | Running workflow 'PythonPipBuilder'
2024-06-30 23:47:04,786 | Running PythonPipBuilder:ResolveDependencies
2024-06-30 23:47:05,000 | calling pip download -r ~/test/lambda/layers/onnx/requirements.txt --dest /tmp/tmpt5hdp82u --exists-action i
2024-06-30 23:47:06,852 | pip stdout: b'Collecting onnxruntime==1.18.1 (from -r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for onnxruntime==1.18.1 from https://files.pythonhosted.org/packages/a3/0a/89bc7acdf7b311ec5cdf6c01983e8ecb23f7b1ba7a1b2d2fd10d33dfd24a/onnxruntime-1.18.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata
 Using cached onnxruntime-1.18.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (4.3 kB)
Collecting coloredlogs (from onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for coloredlogs from https://files.pythonhosted.org/packages/a7/06/3d6badcf13db419e25b07041d9c7b4a2c331d3f4e7134445ec5df57714cd/coloredlogs-15.0.1-py2.py3-none-any.whl.metadata
 Using cached coloredlogs-15.0.1-py2.py3-none-any.whl.metadata (12 kB)
Collecting flatbuffers (from onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for flatbuffers from https://files.pythonhosted.org/packages/41/f0/7e988a019bc54b2dbd0ad4182ef2d53488bb02e58694cd79d61369e85900/flatbuffers-24.3.25-py2.py3-none-any.whl.metadata
 Using cached flatbuffers-24.3.25-py2.py3-none-any.whl.metadata (850 bytes)
Collecting numpy<2.0,>=1.21.6 (from onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for numpy<2.0,>=1.21.6 from https://files.pythonhosted.org/packages/0f/50/de23fde84e45f5c4fda2488c759b69990fd4512387a8632860f3ac9cd225/numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
 Using cached numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Collecting packaging (from onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for packaging from https://files.pythonhosted.org/packages/08/aa/cc0199a5f0ad350994d660967a8efb233fe0416e4639146c089643407ce6/packaging-24.1-py3-none-any.whl.metadata
 Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting protobuf (from onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for protobuf from https://files.pythonhosted.org/packages/27/e4/8dc4546be46873f8950cb44cdfe19b79d66d26e53c4ee5e3440406257fcd/protobuf-5.27.2-cp38-abi3-manylinux2014_x86_64.whl.metadata
 Using cached protobuf-5.27.2-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)
Collecting sympy (from onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for sympy from https://files.pythonhosted.org/packages/61/53/e18c8c97d0b2724d85c9830477e3ebea3acf1dcdc6deb344d5d9c93a9946/sympy-1.12.1-py3-none-any.whl.metadata
 Using cached sympy-1.12.1-py3-none-any.whl.metadata (12 kB)
Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for humanfriendly>=9.1 from https://files.pythonhosted.org/packages/f0/0f/310fb31e39e2d734ccaa2c0fb981ee41f7bd5056ce9bc29b2248bd569169/humanfriendly-10.0-py2.py3-none-any.whl.metadata
 Using cached humanfriendly-10.0-py2.py3-none-any.whl.metadata (9.2 kB)
Collecting mpmath<1.4.0,>=1.1.0 (from sympy->onnxruntime==1.18.1->-r ~/test/lambda/layers/onnx/requirements.txt (line 1))
 Obtaining dependency information for mpmath<1.4.0,>=1.1.0 from https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl.metadata
 Using cachedmpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Using cached onnxruntime-1.18.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.8 MB)
Using cached numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
Using cached coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Using cached flatbuffers-24.3.25-py2.py3-none-any.whl (26 kB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached protobuf-5.27.2-cp38-abi3-manylinux2014_x86_64.whl (309 kB)
Using cached sympy-1.12.1-py3-none-any.whl (5.7 MB)
Using cached humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
Saved
/tmp/tmpt5hdp82u/onnxruntime-1.18.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Saved
/tmp/tmpt5hdp82u/numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Saved
/tmp/tmpt5hdp82u/coloredlogs-15.0.1-py2.py3-none-any.whl
Saved
/tmp/tmpt5hdp82u/flatbuffers-24.3.25-py2.py3-none-any.whl
Saved
/tmp/tmpt5hdp82u/packaging-24.1-py3-none-any.whl
Saved
/tmp/tmpt5hdp82u/protobuf-5.27.2-cp38-abi3-manylinux2014_x86_64.whl
Saved
/tmp/tmpt5hdp82u/sympy-1.12.1-py3-none-any.whl
Saved
/tmp/tmpt5hdp82u/humanfriendly-10.0-py2.py3-none-any.whl
Saved
/tmp/tmpt5hdp82u/mpmath-1.3.0-py3-none-any.whl
Successfully downloaded onnxruntime numpy coloredlogs flatbuffers packaging protobuf sympy humanfriendly mpmath
'
2024-06-30 23:47:06,858 | pip stderr: b"WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
"
2024-06-30 23:47:06,859 | Full dependency closure: {coloredlogs==15.0.1(wheel), flatbuffers==24.3.25(wheel), humanfriendly==10.0(wheel), protobuf==5.27.2(wheel), sympy==1.12.1(wheel), numpy==1.26.4(wheel), packaging==24.1(wheel), mpmath==1.3.0(wheel), onnxruntime==1.18.1(wheel)}
2024-06-30 23:47:06,860 | initial compatible: {coloredlogs==15.0.1(wheel), flatbuffers==24.3.25(wheel), humanfriendly==10.0(wheel), protobuf==5.27.2(wheel), sympy==1.12.1(wheel), numpy==1.26.4(wheel), mpmath==1.3.0(wheel), packaging==24.1(wheel)}
2024-06-30 23:47:06,861 | initial incompatible: {onnxruntime==1.18.1(wheel)}
2024-06-30 23:47:06,862 | Downloading missing wheels: {onnxruntime==1.18.1(wheel)}
2024-06-30 23:47:06,862 | calling pip download --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --implementation cp --abi cp312 --dest /tmp/tmpt5hdp82u onnxruntime==1.18.1
2024-06-30 23:47:07,496 | pip stdout: b''
2024-06-30 23:47:07,496 | pip stderr: b"WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
ERROR: Could not find a version that satisfies the requirement onnxruntime==1.18.1 (from versions: none)
ERROR: No matching distribution found for onnxruntime==1.18.1
"
2024-06-30 23:47:07,497 | compatible wheels after second download pass: {coloredlogs==15.0.1(wheel), flatbuffers==24.3.25(wheel), humanfriendly==10.0(wheel), protobuf==5.27.2(wheel), sympy==1.12.1(wheel), numpy==1.26.4(wheel), mpmath==1.3.0(wheel), packaging==24.1(wheel)}
2024-06-30 23:47:07,499 | Build missing wheels from sdists (C compiling True): set()
2024-06-30 23:47:07,499 | compatible after building wheels (no C compiling): {coloredlogs==15.0.1(wheel), flatbuffers==24.3.25(wheel), humanfriendly==10.0(wheel), protobuf==5.27.2(wheel), sympy==1.12.1(wheel), numpy==1.26.4(wheel), mpmath==1.3.0(wheel), packaging==24.1(wheel)}
2024-06-30 23:47:07,500 | Build missing wheels from sdists (C compiling False): set()
2024-06-30 23:47:07,501 | compatible after building wheels (C compiling): {coloredlogs==15.0.1(wheel), flatbuffers==24.3.25(wheel), humanfriendly==10.0(wheel), protobuf==5.27.2(wheel), sympy==1.12.1(wheel), numpy==1.26.4(wheel), mpmath==1.3.0(wheel), packaging==24.1(wheel)}
2024-06-30 23:47:07,503 | Final compatible: {coloredlogs==15.0.1(wheel), flatbuffers==24.3.25(wheel), humanfriendly==10.0(wheel), protobuf==5.27.2(wheel), sympy==1.12.1(wheel), numpy==1.26.4(wheel), mpmath==1.3.0(wheel), packaging==24.1(wheel)}
2024-06-30 23:47:07,504 | Final incompatible: {onnxruntime==1.18.1(wheel)}
2024-06-30 23:47:07,505 | Final missing wheels: {onnxruntime==1.18.1(wheel)}
2024-06-30 23:47:07,948 | PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
 File "aws_lambda_builders/workflows/python_pip/actions.py", line 71, in execute
 File "aws_lambda_builders/workflows/python_pip/packager.py", line 157, in build_dependencies
 File "aws_lambda_builders/workflows/python_pip/packager.py", line 265, in build_site_packages
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {onnxruntime==1.18.1(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "aws_lambda_builders/workflow.py", line 374, in run
 File "aws_lambda_builders/workflows/python_pip/actions.py", line 77, in execute
aws_lambda_builders.actions.ActionFailedError: {onnxruntime==1.18.1(wheel)}

Build Failed

Что я пробовал

Если я создаю виртуальную среду с Python 3.12 и устанавливаю тот же файл requirements.txt, он устанавливается без проблем.

Я использую SAM CLI, version 1.119.0 и pip 24.1.1.

🤔 А знаете ли вы, что...
Python позволяет создавать сценарии для автоматизации задач и обработки данных.


1
102
1

Ответ:

Решено

У меня была такая же проблема, и я мог решить ее с помощью

  • понижение версии Сэма до версии 1.94.0
  • используя python3.11
  • исправление onnxruntime до версии 1.16.0

Возможно, есть и другие возможные комбинации, но мне подошла эта!