LDAP Group settings make the LDAP queries relative to the "LDAP Setting" that is selected. Thus, the Search Base at the LDAP Group Settings page should be set relative to LDAP URL plus Search Base of LDAP Settings page. For example, if LDAP Settings has LDAP URL as ldap://abc and Search Base as ou=users, then LDAP Group's searches will be made relative to ldap://abc/ou=users .
If you add Search base at the LDAP Group Setting, then the search will be made relative to the combination of the search bases set at LDAP Setting and the LDAP Group Setting. For example, if LDAP Group has ou=groups as a Search Filter and LDAP has ou=users as a Search Filter then Artifactory will search for groups under ou=groups,ou=users