The way in which Visual Studio and other debugging tools match an assembly and PDB file, is by using the assembly hash. This hash is stored in the .dll and .pdb files and must match for the debugging and source stepping to work.
When a .nupkg contains .pdb files, Visual Studio will not reach out to MyGet to download Symbols and sources. When trying to debug using this type of package, Visual Studio will find the .pdb on the disk instead of reaching out to MyGet to download it, and therefore will fail to step into the code because of that.
Prerequisites
In Visual Studio, under Tools | Options (or Debug | Options) | Debugging | General , clear the Enable Just My Code field.
Set Artifactory to be your Symbol Server in Visual Studio by going to Tools | Options | Debugging | Symbols and adding the virtual or local repository URL path.