Without spending too much time on it, I think a cursory check shows git doing "the smart thing" with ignore rules.

  $ mkdir foo
  $ cd foo
  $ mkdir $(seq 1 1000)
  $ git init .
  $ strace -c git status
    On branch master

    No commits yet

    nothing to commit (create/copy files and use "git add" to track)
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
     75.97    0.058830          57      1035        15 openat
     16.48    0.012764           6      2002           getdents
      3.87    0.002999           3      1021           close
      2.91    0.002254           2      1022           fstat
      0.19    0.000147           8        18        14 lstat
      0.13    0.000100           4        24           read
      0.12    0.000092           6        16        12 stat
      0.09    0.000072           3        21        14 access
      0.06    0.000048          10         5           write
      0.04    0.000033          33         1           unlink
      0.03    0.000020           3         8           rt_sigaction
      0.02    0.000018           2        12           mprotect
      0.02    0.000015          15         1           munmap
      0.01    0.000010           3         4           getcwd
      0.01    0.000006           0        16           mmap
      0.01    0.000006           6         1           ioctl
      0.01    0.000005           2         3           brk
      0.01    0.000004           4         1           chdir
      0.00    0.000003           2         2           getpid
      0.00    0.000003           3         1         1 readlink
      0.00    0.000002           1         2           rt_sigprocmask
      0.00    0.000002           2         1           set_tid_address
      0.00    0.000002           2         1           set_robust_list
      0.00    0.000002           2         1           prlimit64
      0.00    0.000000           0         1           execve
      0.00    0.000000           0         1           arch_prctl
    ------ ----------- ----------- --------- --------- ----------------
    100.00    0.077437                  5221        56 total

Now let's add an ignore-all and check:

  $ echo "/*" > .gitignore
  $ strace -c git status
    On branch master

    No commits yet

    nothing to commit (create/copy files and use "git add" to track)
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
     46.25    0.000568         284         2           getdents
     16.78    0.000206           6        35        14 openat
      7.41    0.000091           6        16        12 stat
      7.17    0.000088           4        21        14 access
      5.62    0.000069           3        25           read
      4.72    0.000058           3        18        14 lstat
      4.15    0.000051           2        22           close
      2.93    0.000036          36         1           unlink
      2.28    0.000028           1        23           fstat
      0.81    0.000010           3         4           getcwd
      0.65    0.000008           1         8           rt_sigaction
      0.41    0.000005           5         1           ioctl
      0.33    0.000004           4         1           chdir
      0.24    0.000003           2         2           getpid
      0.24    0.000003           3         1         1 readlink
      0.00    0.000000           0         5           write
      0.00    0.000000           0        16           mmap
      0.00    0.000000           0        12           mprotect
      0.00    0.000000           0         1           munmap
      0.00    0.000000           0         3           brk
      0.00    0.000000           0         2           rt_sigprocmask
      0.00    0.000000           0         1           execve
      0.00    0.000000           0         1           arch_prctl
      0.00    0.000000           0         1           set_tid_address
      0.00    0.000000           0         1           set_robust_list
      0.00    0.000000           0         1           prlimit64
    ------ ----------- ----------- --------- --------- ----------------
    100.00    0.001228                   224        55 total