Wikipedia Image Scraper

I've written a "Wikipedia Featured Pictures" scraper in PowerShell, using DotNetWikiBot. It downloads all 2500+ featured images (featured images are the images that are shown on the front page of Wikipedia).

$LocalFolderToSaveTo = "C:\DownloadLocation"
[System.Reflection.Assembly]::LoadFile("D:\Documents\WindowsPowerShell\test\dotnetwikibot\DotNetWikiBot.dll")
$EnWiki = new-object DotNetWikiBot.Site("http://en.wikipedia.org", "WikipediaUserName", "WikipediaPassword")
$PageList = new-object DotNetWikiBot.PageList($enWiki);
$PageList.FillFromCategory("Featured pictures")
$ImageExts = ".jpg", ".png", ".jpeg"
Foreach($Page in $PageList.pages)
{
if(($ImageExts -eq $Page.Title.substring($Page.Title.length - 4, 4)) -and (!($Page.Title.startswith("File talk"))) )
{
$DownloadTo = $Null
$RenamedPage = $Page.Title.replace("File:", "")
$RenamedPage = $RenamedPage.replace("`"", "") # "
$RenamedPage = $RenamedPage.replace("`'", "")
$DownloadTo = join-path $LocalFolderToSaveTo ($RenamedPage)
$page.DownloadImage($DownloadTo)
Start-Sleep 1 #Build in delay to prevent hammering and/or Ban.
}
}

Text/HTML

There is some code out there that shows how to do defrag hard disks using WMI. Basically:

 

string]$query = 'Select * from Win32_Volume where DriveType = 3'
$volumes = Get-WmiObject -Query $query -ComputerName $ComputerName
$result = ($volume.Defrag($force)).ReturnValue

Of course, this can take quite some time when running this against a list of servers, since .Defrag() does not start on the second computer, until it completed the first. You can avoid this (assuming you have PowerShell 2), by using start job:

start-job -Name $ComputerName `
    -initializationScript {ipmo defrag} `
    -scriptblock {Param ($ComputerName) Start-CustomDefrag $ComputerName} `
    -ArgumentList $ComputerName

Save this as UserFolder\WindowsPowerShell\modules\Defrag\Defrag.psm1:


#=====================================================================
# Start-CustomDefrag
#=====================================================================
function Start-CustomDefrag {
<#

.SYNOPSIS
Defragments the targeted computers hard drives.

.DESCRIPTION

Returns: The result description for each drive.

.PARAMETER drive
Drive to defrag

.PARAMETER force
Optional
Wether to do a forced defrag or not.

.EXAMPLE

Start-CustomDefrag -ComputerNames "servername1" -verbose

.EXAMPLE

Start-CustomDefrag -ComputerNames ("server2", "server2") -verbose

.NOTES

.LINK

http://winfred.com
#>
param(
[Parameter(Position=0, Mandatory=$True)]$ComputerNames,
[string]$drive,
[switch]$force
)
    [string]$query = 'Select * from Win32_Volume where DriveType = 3'
    if ($drive) 
    {
        $query += " And DriveLetter LIKE '$drive%'"
    }
    
    Foreach($ComputerName in $ComputerNames)
    {
        $volumes = Get-WmiObject -Query $query -ComputerName $ComputerName

        foreach ($volume in $volumes)
        {
            "$(Get-date) - $ComputerName - $($volume.Name)" | Out-File (join-path $PsScriptRoot Log.txt) -append
            $result = ($volume.Defrag($force)).ReturnValue
            switch ($result) 
            {
              0  {$ReturnMessage =  'Success'}
              1  {$ReturnMessage =  'Access Denied'}
              2  {$ReturnMessage =  'Not Supported'}
              3  {$ReturnMessage =  'Volume Dirty Bit Set'}
              4  {$ReturnMessage =  'Not Enough Free Space'}
              5  {$ReturnMessage =  'Corrupt MFT Detected'}
              6  {$ReturnMessage =  'Call Cancelled'}
              7  {$ReturnMessage =  'Cancellation Request Requested Too Late'}
              8  {$ReturnMessage =  'Defrag In Progress'}
              9  {$ReturnMessage =  'Defrag Engine Unavailable'}
              10 {$ReturnMessage =  'Defrag Engine Error'}
              11 {$ReturnMessage =  'Unknown Error'}
            }
        }
    }
    "$(Get-date) - $ComputerName - $ReturnMessage" | Out-File (join-path $PsScriptRoot Log.txt) -append
}

#=====================================================================
# Start-DefragVMs
#=====================================================================
Function Start-DefragVMs
{
<#

.SYNOPSIS

    Start-DefragVMs

.DESCRIPTION

The number specified in -VMNumberToDefrag is the VM on each server to defrag.
If a server has 3 VMs, -VMNumberToDefrag means the first VM in the list, 2, the second etc.
-HyperVSubset is to perform a match against a subset of hyper-V servers, to exclude certain hosts
For example when these hosts are actively migrating VMs.

.EXAMPLE

    Start-DefragVMs -VMMServer "VMMServer1" -VMNumberToDefrag 1 -HyperVSubset "Servers1"
    

.NOTES
    Needs the clixml export located in: "\\VMMServer\c$\VMMstatistics\ExportedObjects.xml"
    
    Please remember that start-job conflicts with start-job of the VMM snapin.
    To avoid this, use powershell -noprofile to run the defrag job.
    To see which jobs are still running, run:
    foreach($Job in (get-job)){write-host "$($Job.Name) - $($Job.State)" -fore cyan}
.LINK

http://winfred.com

#>
param(
[Parameter(Position=0, Mandatory=$True)]$VMMServer,
[Parameter(Position=1, Mandatory=$True)]$VMNumberToDefrag,
[Parameter(Position=2, Mandatory=$False)]$HyperVSubset
)
    $ComputerName = $Null
    $ObjectName =  import-clixml "\\$($VMMServer)\c$\VMMstatistics\ExportedObjects.xml"
    foreach($Row in ($ObjectName | where {$_.VMhostInfo.Name -match $HyperVSubset}))
    {
        $i = 0
        foreach($VMInfoObject in ($Row.VMInfo))
        {
            if($VMInfoObject.Status -eq 0)
            {
                $i++
                if($i -eq $VMNumberToDefrag)
                {
                    $ComputerName = $VMInfoObject.Name
                    start-job -Name $ComputerName `
                        -initializationScript {ipmo defrag} `
                        -scriptblock {Param ($ComputerName) Start-CustomDefrag $ComputerName} `
                        -ArgumentList $ComputerName
                }
            }
        }
    }
}

#=====================================================================
# Test-ForDefragStart
#=====================================================================
Function Test-ForDefragStart
{
<#

.SYNOPSIS

.DESCRIPTION

.EXAMPLE

Test-ForDefragStart "ServerName1" -verbose

.NOTES

.LINK

http://winfred.com
#>
param(
[Parameter(Position=0, Mandatory=$True)]$ComputerName
)
    $DefragStarted = $False
    get-eventlog -computername $ComputerName -logname "SYSTEM" -Newest 10 | % `
    {
        if(($_.EventID -eq "7036") -and ($_.message -eq "The Disk Defragmenter service entered the running state."))
        {
            write-verbose "$ComputerName $($_.message)"
            $DefragStarted = $True
        }
    }
    if($DefragStarted)
    {
        Return $True
    }else{
        Return $False
    }
}
{/codecitation}

 Run it like this:

Import-Module Defrag

get-help Start-DefragVMs -full

get-help Start-CustomDefrag -full