Forum Discussion

kzng's avatar
kzng
Copper Contributor
Mar 13, 2025

Issue with Auto Setting for Copy Parallelism in ADF Copy Activity

Hello everyone,

I've been utilizing Azure Data Factory (ADF) and noticed the option to set the degree of copy parallelism in a copy activity, which can significantly enhance performance when copying data, such as blob content to an SQL table.

However, despite setting this option to "Auto," the degree of parallelism remains fixed at 1. This occurs even when copying hundreds of millions of rows, resulting in a process that takes over 2 hours. My Azure SQL database is scaled to 24 vCores, which should theoretically support higher parallelism.

Am I missing something, or is the "Auto" setting for copy parallelism not functioning as expected?

Any insights or suggestions would be greatly appreciated!

Thank you.

azure data factory
Copy Activity

1 Reply

Sort By
  • MoritzG's avatar
    MoritzG
    Copper Contributor
    Mar 17, 2025

    Hello,

    not sure if I can help, only some insights:

    In my experience, the "Auto" option does not always maximize parallelism effectively. It often defaults to conservative/safe settings to avoid overwhelming the target system.

    Also, if you copy from blob storage, you need to have the data split into multiple files so that it can be read in parallel. And maybe try to scale the DIUs up.

    For more insights maybe enable logging in the ADF.

    Best regards!

Resources