Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] Running Windows 10 in both bare-metal and KVM/QEMU
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Anton Gubarkov
n00b
n00b


Joined: 13 Mar 2008
Posts: 65

PostPosted: Sat Nov 11, 2017 6:54 pm    Post subject: [SOLVED] Running Windows 10 in both bare-metal and KVM/QEMU Reply with quote

I wanted to make use of my Windows 10 OEM installed on the HDD of my notebook. My main system is Gentoo (90% of time). I run several systems (both linux and Windows) as QEMU/KVM guests. Having to reboot into my Windows 10 every time I need an app from that system, had always been a nuisance and I wanted to make it possible to run this Windows install in either bare metal or as QEMU/LVM guest at my discretion. MS EULA allows that, however MS techs have put as many traps on this way as possible.
I assume that you've mastered dual-boot already and can boot to Gentoo and Windows alternately. If not, please tend to this problem first and then return here.


Usually there are 2 caveats that must be solved:

  • Inaccessible boot device
  • Activation


Let's start with Activation. I use rather latest versions of qemu and the kernel: 4.13.12-ck-zfs and QEMU emulator version 2.10.0. The domain xml exerpts that I'll post here may work differently on earlier versions of kernel/qemu.

To trick Windows activation into believing that you haven't changed your hardware a lot, we need to translate parts of the acpi information from the host to the guest.

So I started with a freshly created VM in libvirt/wirt-manager. I run UEFI firmware on the host, so make sure you create your machine in UEFI as well. It will be very difficult to change afterwards. Make arbitrary small disk with SATA bus. We'll discard this disk later anyway.

After that we need to collect a bit of the information about the host. Run:
Code:

r9-008cln ~ # dmidecode -t 1
# dmidecode 3.1
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.

Handle 0x000C, DMI type 1, 27 bytes
System Information
        Manufacturer: LENOVO
        Product Name: 20BEA008RT
        Version: ThinkPad T540p
        Serial Number: R9008CLN
        UUID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
        Wake-up Type: Power Switch
        SKU Number: LENOVO_MT_20BE
        Family: ThinkPad T540p

You need the UUID part of the output. I masked my value with XXs. This is the UUID of the system that must end up in the <uuid/> tag of the libvirt vm definition.

Identify the disk that has your Windows 10 install.
Code:

r9-008cln ~ # blkid|grep -i ntfs
/dev/sda6: UUID="4A88979488977D5F" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="132861f0-01e0-4e25-860e-4d55192f8659"

I have only one partition with ntfs. So I have /dev/sda as the disk. These names can change - when you connect more devices to your system. To make sure your VM access always the same disk, use
Code:

9-008cln ~ # ls -l /dev/disk/by-id
total 0
lrwxrwxrwx 1 root root  9 Nov 11 19:53 ata-HL-DT-ST_DVDRAM_GU70N_M2LDC280636 -> ../../sr0
lrwxrwxrwx 1 root root  9 Nov 11 20:01 ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997 -> ../../sda
lrwxrwxrwx 1 root root 10 Nov 11 20:01 ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Nov 11 20:01 ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 Nov 11 20:01 ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997-part3 -> ../../sda3
lrwxrwxrwx 1 root root 10 Nov 11 20:01 ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997-part4 -> ../../sda4
lrwxrwxrwx 1 root root 10 Nov 11 20:01 ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997-part5 -> ../../sda5
lrwxrwxrwx 1 root root 10 Nov 11 20:01 ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997-part6 -> ../../sda6

I can see that /dev/disk/by-id/ata-ST2000LM003_HN-M201RAD_S32WJ9CF159997. Note this path down for later insertion into VM definition.

Let's find out the HDD's serial number. Windows also checks this number for machine signature.
Code:

r9-008cln ~ # smartctl -i /dev/sda
smartctl 6.6 2017-11-05 r4594 [x86_64-linux-4.13.12-ck-zfs] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Samsung SpinPoint M9T
Device Model:     ST2000LM003 HN-M201RAD
Serial Number:    S32WJ9CF159997
LU WWN Device Id: 5 0004cf 20c4d1e0e
Firmware Version: 2BC10002
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Form Factor:      2.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 6
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Nov 11 21:17:13 2017 MSK
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Note down the Serial Number from the output.

Now we've collected enough information to edit our VM definition in the text editor.

Code:
virsh edit <your-machine-name-here>

Make the edits below (variable names are in []. replace with your values removing [])
Change the <uuid> to the value noted from dmidecode. Change the machine name too (<name> tag) - or libvirt will not let you save.
Code:

<domain type='kvm'>
  <name>[change to whatever new name]</name>
  <uuid>[uuid from dmidecode]</uuid>


My PC is rather old, it came with pre-installed and pre-activated Windows 7, so my acpi has a table with the public key of this activation. I decided to export this information to the guest as well. I can't ensure it is need. You can have a look at the table yourself running a command. (I masked some lines for security reasons). Your table may be almost empty (contain only zeros).
Code:

r9-008cln ~ # hexdump /sys/firmware/acpi/tables/SLIC
0000000 4c53 4349 0176 0000 1e01 454c 4f4e 4f56
0000010 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
0000020 0001 0000 0000 0000 009c 0000 0206 0000
0000030 2400 0000 5352 3141 0400 0000 0001 0001
0000040 1669 9f4a 4bb1 fb3a 2080 afaa f9c4 c13e
0000050 4980 6aee 2665 1e72 bfcd 2f5f d696 0ac0
0000060 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
0000070 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
0000080 9515 faf9 dcfd b705 674d 2d7f 84b3 2033
0000090 d1e1 2a79 6aa7 d177 20b6 762a c542 e9d5
00000a0 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
00000b0 ec07 1a7b a129 f1c1 fd91 8648 3e6e cbce
00000c0 0001 0000 00b6 0000 0000 0002 454c 4f4e
00000d0 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
00000e0 2053 0001 0002 0000 0000 0000 0000 0000
00000f0 0000 0000 0000 c697 1bce 210e 888c 0dc8
0000100 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
0000110 3658 3e18 ce68 07da 75d2 4719 0d9b aba4
0000120 4e80 1fbf 9b16 a811 f939 cfa9 8f30 1fdf
0000130 fded 9b6e 4f25 86b3 80af 29a3 4bf8 e872
0000140 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
0000150 b996 7750 58a5 d5d6 daed 437f 67ab ceeb
0000160 XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX
0000170 8c3a 8839 9e1b
0000176


The qemu starts under libvirt as user qemu and this user has no access to /sys/firmware/acpi/tables/SLIC, so you need to make a copy of this file and change ownership to the user you run qemu as. Note down the full file name (preferably in the /var/lib/libvirt/qemu/nvram)

Locate your <os> tag and add the follwing part to it. It will export host computer information and the table we've copied above.
Code:

 <os>
    .... some tags here
    <acpi>
      <table type='slic'>/var/lib/libvirt/qemu/nvram/SLIC</table>
    </acpi>
    <smbios mode='host'/>
  </os>


Locate <disk> tag that you've created. :
Code:

    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='directsync' io='native'/>
      <source dev='/dev/disk/by-id/[insert disk id noted above]'/>
      <target dev='sdd' bus='sata'/>
      <serial>[serial from smartctl]</serial>
      ....some tags removed...
    </disk>


Optional: I have a hiDPI display and the default 16MB of video memory limits my available resolutions. So I change vgamem to 65536:
Code:

    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>


So we are done with editing, you may save. BUT DON'T RUSH TO BOOT YOUR NEW SHINY GUEST YET. We need to cater to infamous inaccessible boot device stop error.
Windows 10 introduced a new behaviour to block boot loading any storage device drivers except the one used to access the Windows partition in this boot. Nice? What a care about us, users. Methods described for Windows 7 no longer work.

Now, we need to reboot into Windows 10 bare metal and login as an adminstrator. I'll get back in the next post.
Back to top
View user's profile Send private message
Anton Gubarkov
n00b
n00b


Joined: 13 Mar 2008
Posts: 65

PostPosted: Sat Nov 11, 2017 7:36 pm    Post subject: Fight with storage controllers Reply with quote

Run Device Manager and switch to view by connections. Find storage controller that connects your drive where Windows 10 is installed. Inspect the drives and access the Volumes tab to find the correct drive and controller.

Once the controller is identified, you need to open its properties and look up the driver details. Mine is C:\WINDOWS\System32\DRIVERS\iaStorA.sys. Yours may be different, note it down. You can close the device manager now.

If your storage controller is storahci.sys you can skip the remaining section about storage controllers.

Open the registry editor and open the key [HKLM\SYSTEM\ControlSet001\Services] and locate the key [storahci\StartOverride]. There is a DWORD parameter of 0 in it with value 3 that means - override startup of this driver as non-essential for Windows 10 boot. Edit this value to 0 and save. You'll need to do it before every shut down because Windows 10 will reset it to 3 after every startup using another storage controller. Yes, it is annoying, I know. Thanks Microsoft for one more trap on our journey. I'll tackle automation in the next post.

This may be enough for your Windows 10 to start in the qemu guest.
Optional: disable fastboot and hibernation. Fastboot is a boot using hibernated state. If you try to resume from hibernation in the environment different from the one you hibernated in, the results are unpredictable.

I assume that your windows 10 is activated now. If not, tackle this problem before going on further.
Oh! one last warning - make sure your have a full backup of your data now and you have a recovery drive to re-install your Windows 10 if things go awfully wrong.

Reboot into Gentoo and prepare to launch your new shiny guest VM with Windows 10.
Back to top
View user's profile Send private message
Anton Gubarkov
n00b
n00b


Joined: 13 Mar 2008
Posts: 65

PostPosted: Sat Nov 11, 2017 8:11 pm    Post subject: Automating StartOverride Reply with quote

Now start your new guest in libvirt virt-manager and enjoy your success! Verify, that your Windows 10 is still activated. Mine was. You yours does not, don't blame me. You have ben warned!
If you hit inaccessible boot device, you'll need to get back to Windows 10 bare metal and perform safeboot next time:

Open admin command prompt and type:
Code:
bcdedit /set {current} safeboot minimal


This command sets additional parameter to the windows boot manager entry to force safe mode boot loading (with all storage drivers enabled and ready). I couldn't make my guest boot with this command alone. I still needed to set storahci\StartOverride\0 from 3 to 0.

Note down the command to remove this parameter. You need a local admin account to do it. You may not be able to login with Microsoft account due to unavailbale network.
Code:
bcdedit /deletevalue {current} safeboot


To boot back into bare-metal, you need to reset StartOverride\0 parameter for the storage driver used to boot your bare-metal environment before shutting down your guest.
I suggest you test reboots between VM and bare metal several times to make sure everything works as expected.

To cope with the nuisance of manual resets I searched the internet hi and lo but couldn't find any parameters to stop this behaviour. So I wrote a small powershell script. I kidnly ask the community for a better solution.

Code:

# get a list of storage drivers as parameters

if ($args.Count -eq 0) {
    echo "Must indicate at least 1 storage driver name"
    exit 1
}
$rootPath = "HKLM:\SYSTEM\ControlSet001\Services\"
$tailPath = "\StartOverride"

foreach ($p in $args) {
    echo ("Checking: "+$rootPath+$p+$tailPath)
    if (Test-Path ($rootPath+$p+$tailPath)) {
        echo ($rootPath+$p+$tailPath) "exists"
        Set-ItemProperty -Name "0" -Path ($rootPath+$p+$tailPath) -Value 0
    }
}


Save this script into $WINDIR\System32\GroupPolicy\Machine\Scripts\Startup under any descriptive name and .ps1 extension.

Run gpedit.msc to edit the group policy for Computer->Scripts and add a powershell startup script. You want it at startup to protecet yourself from unclean shutdowns. Browse for the saved script file name and put the following parameters:
Code:

-ExecutionPolicy BYPASS storahci vioscsi iaStorA

-ExecutionPolicy BYPASS allows unrestricted script execution, and the list of storage driver names follows to be unlocked for boot on the next startup.
Apply the policy and close the editor. DO NOT configure a priority for this group policy. Doing so prevents it from execution.
Reboot the VM to apply the policy. Verify that the StartOverride\0 parameters are reset to 0. Debug your setup if they are not.

This concludes my solution to the problem. I welcome your comments and experiences.
Back to top
View user's profile Send private message
Crocodillian
n00b
n00b


Joined: 16 Oct 2018
Posts: 3
Location: San Francisco, CA, United States

PostPosted: Tue Oct 16, 2018 9:53 am    Post subject: got this working, thank you Reply with quote

So I followed this guide, initially I got the disk device wrong because I misread and used the partition device instead of the disk device, but then I switched to the disk device and everything works fine.

I also installed this to unmount the ESP when the VM is booting:

https://gitlab.com/ranolfi/rvirtesp

I did not have a mobo UUID on this supermicro mobo, at least dmidecode reports it as "not settable".

I did not have a SLIC either, since this is a workstation/server board and not e.g. a laptop.

So activation did not work at first, probably because of this, I did copy the hard drive serial btw.

I saw this SU thread:

https://superuser.com/questions/1057959/windows-10-in-kvm-change-boot-disk-to-virtio

which says virtio is faster for raw disks, so I switched it to that.

It also says that if activation fails, and then you boot the real hardware followed by the VM in sequence it will be activated properly, I did this and it worked!

I installed your powershell script as described, don't know if I need it or not.. My real hardware controller is stornvme.

Thanks for the guide, it was very helpful!
Back to top
View user's profile Send private message
ml35
n00b
n00b


Joined: 29 Nov 2019
Posts: 2

PostPosted: Fri Nov 29, 2019 10:49 am    Post subject: Re: Automating StartOverride Reply with quote

Anton Gubarkov wrote:
[...]
Save this script into $WINDIR\System32\GroupPolicy\Machine\Scripts\Startup under any descriptive name and .ps1 extension.

Run gpedit.msc to edit the group policy for Computer->Scripts and add a powershell startup script. You want it at startup to protecet yourself from unclean shutdowns. Browse for the saved script file name and put the following parameters:
Code:

-ExecutionPolicy BYPASS storahci vioscsi iaStorA

-ExecutionPolicy BYPASS allows unrestricted script execution, and the list of storage driver names follows to be unlocked for boot on the next startup.
Apply the policy and close the editor. DO NOT configure a priority for this group policy. Doing so prevents it from execution.
Reboot the VM to apply the policy. Verify that the StartOverride\0 parameters are reset to 0. Debug your setup if they are not.

This concludes my solution to the problem. I welcome your comments and experiences.


This is gold!
Dear Anton, thank you so much for sharing this. I created an account even if I'm not a gentoo user, just to thank you!

Here is my small observation: on my Windows 1909 OEM installation, I had to make this modification to make it work:

first, I didn't know wtf it is not the powershell script running at startup, so I googled and added this code to log it's output

(add this at the beginning of code)
Code:
$ErrorActionPreference="SilentlyContinue"
Stop-Transcript | out-null
$ErrorActionPreference = "Continue"
Start-Transcript -path C:\kvm-script.txt -append


(add this at the end)
Code:
Stop-Transcript


second, I discovered I have a special "powershell scripts" tab in group policy editor, it turned out the script had to be specified there instead of the classic scripts tab, even if on disk it's the same folder path.

third, IIRC just having execution policy bypass as an argument to ps1 script didn't have an effect for me, so I had to change machine execution policy via gpedit.msc (it didn't allow me to do it from command line)
go to gpedit.msc - computer - administrative - windows components - windows powershell and change "turn on script execution" to "allow local and remote" screenshot - note this is NOT the same as the one setting you change from windows settings in developer menu - this is for running scripts at logon, which is governed by a different policy than the one in windows settings!

here's how it looks on my system. it's working perfectly. thank you so much!
Back to top
View user's profile Send private message
ml35
n00b
n00b


Joined: 29 Nov 2019
Posts: 2

PostPosted: Fri Nov 29, 2019 11:17 am    Post subject: Re: got this working, thank you Reply with quote

Crocodillian wrote:

I installed your powershell script as described, don't know if I need it or not.. My real hardware controller is stornvme.


it is really needed or else the next boot on hw platform change windows will bsod.
Back to top
View user's profile Send private message
Marlo
Veteran
Veteran


Joined: 26 Jul 2003
Posts: 1591

PostPosted: Sun Dec 01, 2019 9:12 pm    Post subject: Reply with quote

Anton Gubarkov!

Thank you very much for this really valuable explanation.
It was not easy for me, but I can start Win10 and have sound.

Only the video resolution remains at 800x600. Whether QXL, VGA or VIRTIO I can not change the resolution.

Is there a hint that could help?
Thanks again
Ma

Solved
I have created a new drive with libvirt and connected to this iso.
https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso

The virtio-win.iso contains drivers, including the QXL driver for the monitor / video. The resolution now goes up to 4096x2160.
I think that's enough.
_________________
------------------------------------------------------------------
http://radio.garden/
Back to top
View user's profile Send private message
Moso
n00b
n00b


Joined: 03 Mar 2020
Posts: 1

PostPosted: Tue Mar 03, 2020 12:14 am    Post subject: Reply with quote

Hi Anton!

Can you please take a look at "Raw vmdk booting of Windows 10 with winload.exe error 0xc000000e - used to work... :(" post and give me some advice?

I tried to understand and adapt your solution but had no success.

I think the problem is almost the same you solved.

:wink:
Back to top
View user's profile Send private message
pmatti
n00b
n00b


Joined: 18 May 2020
Posts: 1

PostPosted: Mon May 18, 2020 8:21 am    Post subject: Thanks this was exactly what I needed Reply with quote

The critical parts for me were:
- Set the vm uuid
- Set the disk serial number
- Use a whole disk, not a partition
- Disable fastboot.

That was enough to get my bare-metal windows machine on an sdd into a qemu-virtualized one, and to have it authenticate

Thanks so much.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum