Microsoft TechNet Blogs

ForEach is an unusual statement. Depending on the context in which it's used, sometimes it's an alias of ForEach-Object, but sometimes it is not. The following article has more details on this, but one thing that I'd like to highlight now is that one way that ForEach and ForEach-Object can differ from each other is in memory use, as ForEach will load all objects that it's iterating over into memory first before starting the loop, whereas ForEach-Object will not. As suggested before, more details can be found here:

https://blogs.technet.microsoft.com/heyscriptingguy/2014/07/08/getting-to-know-foreach-and-foreach-object/

Going back to the naming conventions, the conventional way to write ForEach is in PascalCase. The variations slip into code as it's also an alias, and aliases tend to be in all lower case, and also because naming convention guidelines are just suggestions, not something that is strongly enforced (Foreach does seem to be commonly used, even if it's not necessarily in the standard styles).

Generally speaking, PowerShell coding style is similar to .NET coding style. You can read a discussion about it here:

https://github.com/PoshCode/PowerShellPracticeAndStyle/issues/36

By the way, I respect your right to use whatever language you like best, I'm just clarifying that whilst PowerShell may not have it's "PEP-8" or gofmt, there are still conventions that PowerShell coders follow.

Here, yes, I agree that the foreach statements would be better, possibly even significantly, but in a general case the difference is usually not worth bothering with.

While ForEach-Object is like an order of magnitude slower, we're talking like 100 ms vs 10 ms to iterate through 10,000 items. (https://blogs.technet.microsoft.com/heyscriptingguy/2014/07/08/getting-to-know-foreach-and-foreach-object/) In other words, you have to have a good number of objects to make foreach meaningfully faster that when you do, you often find that either a) lose time to powershell.exe allocating more system memory to store the collection, or b) the loop itself is two or three orders of magnitude slower already so it's ultimately a trivial optimization.

Also, the foreach statement waits to allocate the entire set to memory ($y in your example) while ForEach-Object begins processing as soon as the first object comes through the pipeline. If you're using the output of a command for $y that returns output with some lag, you might find that ForEach-Object actually turns out to be faster because the first objects start being processed immediately. If you're returning data over a network or dealing with particularly large objects, ForEach-Object can be better.

Finally, the foreach statement itself doesn't work with pipelines either as input or output, so it's not appropriate for a lot of scenarios.

The article confuses address space and memory reservations as you note. It again makes the confusion when it suggests the virtual size column in Process Explorer is a reflection of the address space[1]. It also suggests that working set, private bytes, and committed memory are the same thing[2]. The article does not even live up to its click-baity title: Task Manager's default memory column of "private working set" is a decent measure of how much memory a process is uniquely using and task manager has the ability to add all the other measures of memory usage that the article mentions.

For a better explanation of virtual memory in Windows, I recommend Mark Russinovich's article[3]. His tool VMMap[4] is useful for visualizing the memory usage of an individual process.

[1]: The reserved memory is really large for 64-bit processes that use Control Flow Gaurd: http://www.alex-ionescu.com/?p=246

[2]: Task Manager and Process Explorer add to the confusion by calling the same memory different things (Process Explorer's "private bytes" number is the same as Task Manager's "commit size" number on Windows 10 1809).

[3]: https://blogs.technet.microsoft.com/markrussinovich/2008/11/17/pushing-the-limits-of-windows-virtual-memory/

[4]: https://docs.microsoft.com/en-us/sysinternals/downloads/vmmap

Reminds me of how Windows Vista's "Multimedia Class Scheduler Service" would put a low cap on network throughput if any sound was playing:

http://www.zdnet.com/article/follow-up-playing-music-severely-degrades-network-transfer-performance-in-vista/

Mark Russinovich justified it by explaining that the network interrupt routine was just too expensive to be able to guarantee no glitches in media playback, so it was limited to 10 packets per millisecond when any media was playing:

https://blogs.technet.microsoft.com/markrussinovich/2007/08/26/vista-multimedia-playback-and-network-throughput/

but obviously this is a pretty crappy one-size-fits-all prioritization scheme for something marketed as a most-sophisticated best-ever OS at the time:

https://blogs.technet.microsoft.com/markrussinovich/2008/02/04/inside-vista-sp1-file-copy-improvements/

Many people had perfectly consistent mp3 playback when copying files over the network 10 times as fast in other OSes (including Win XP!)

Often a company will have a "sophisticated best-ever algorithm" and then put in a hacky lazy work-around for some problem, and obviously don't tell anyone about it. Sometimes the simpler less-sophisticated solution just works better in practice.

I think a lot of the hate on powershell is that it chooses to do things very similar to existing languages, but just different enough to confuse practitioners of anything else.

Posh also suffers in its somewhat easy to invoke any .NET library, so people start comparing it to a more tooled programming language or something with more sane libraries, and the "ISE" is nowhere close to an IDE (why isnt powershell a first class citizen in visual studio! yes I know there are tools, they are not great.)

Nonetheless, after a few years of working in sql server I have adopted more and more posh in my life because its simply the fastest way to get things done if you are doing windows server management.

I usually find myself on one of these two places if I am looking for something:

Mike Fal (SQL Server focus, he is great) http://www.mikefal.net/

Hey Scripting Guy! https://blogs.technet.microsoft.com/heyscriptingguy/

edit: forgot to mention that MSFT helped to cause problems by massive changing how the language did things over time(usually for the better), so ps1 vs ps4 is crazy different, another reason to be annoyed because which version comes with which product is a fun skew to deal with (thanks sqlps having ps2.0!)

Another data point on Anaconda from microsoft's Azure ML team - we've been using Anaconda for the past few years and have been very happy with it. One key feature we like is that we can deploy a nearly identical suite of pkgs on Windows, Linux and MacOS w/o having to worry about mismatched versions, installation/update issues, etc.

Case in point - the Azure ML Jupyter notebooks run on a farm of Linux/docker machines:

https://blogs.technet.microsoft.com/machinelearning/2016/02/10/python-notebooks-in-the-cortana-analytics-gallery/

Once a model is built, it can be deployed for production/scale to the Azure ML backend which runs entirely on a farm of Windows machines.

Both environments have matching Anaconda distros running underneath which makes running on two different OS's practically a non-issue. In fact 95% of our users probably have no idea their notebooks & production site run two OS's (in large part thanks to Python, Linux, Docker, Anaconda, ... and lots of other great open source software).