Cloud, Computer, Internet, Linux

Hi Hadoop!

Heard Hadoop for many times, never got deep into it. Now is a chance, so I started the experiment in Microsoft Azure.

First, you can find a number of Hadoop packages in Azure Marketplace. I chose Hortonworks Sandbox with HDP 2.5.  I tried Hadoop by Bitnami as well, but it’s usability is a bit tricky, I couldn’t find a way to make Bitnami work without creating a number of accounts and expose more of my own information. I may try it later (and enable the boot diagnostic to find the password in the log when the image starts the first time) when I have time. For now, I stick to Hortonworks.

image

Then just follow the standard Azure procedure –

  • Basics: filling in the VM name, username, SSH key or password, subscription, resource group, location, etc.
  • Size: choose the size of the VM.
  • Settings: choose the storage, network, etc. I suggest to leave boot diagnostics enabled.
  • Confirm on the Summary and Buy.

Notice that on the price page, there is warning on the charge besides the Azure VM itself, also since the HDP Sandbox just showed 0.0000 CAD/hr, I don’t think you need to worry too much about it. BTW, Bitnami’s Hadoop is also free, explicitely mentioned.

image

Wait for a few minutes until the deployments succeeds. You can then check the status of your new Hadoop VM.  Hortonworks suggests that you make the public IP static. You can find more detail information on its tutorial page.

Next is to configure your SSH client. I am using PuTTY on Windows, so there are more mouse-clicks than the config example given in the tutorial.  Basically these settings let you connect to your VM in the Azure cloud using various ports from localhost to the remote VM via the SSH tunnel you set up here.

image

Here is how to configure PuTTY:

  • Fill in the public IP of your Hadoop VM
  • Expand Connection – SSH
  • Click Tunnels
  • Fill the source and destination, then click Add button

image

According to the document, you need to add 8 forward ports

image

So in PuTTY, you can add one by one, it should eventually look like this (scroll up and down to see total 8 lines/ports).

image

You can then go back to the Session page, give a name in “Saved Sessions” and save the configuration. Next time, you only need to load it from there.

One trick is that the VM need some time to start and become stable. My first few login attempts failed, only after 20 or 30 minutes can I eventually login. so be patient. After login, you should be able to see the following directories.

image

Then according to the tutorial, keep the SSH session active, you can use brower to visit this page on your VM.

image

Click on the left icon, you will see the dashboard.

image

Click on the right, you can read more advanced topics including the default username and password, and how to change them.

That is the first step into Hadoop.

Mobile

BlackBerry Classic couldn’t start any apps – resolved by soft reset

This started during my trip to UK.  My 22 months old BlackBerry Classic gave me weird message when I wanted to take pictures in the Elephant restaurant in Edinburgh.  I had Maps.me running at the time, and when I clicked the camera icon, it said cannot start camera app, not the exact words but basically refused to start the app.

The first thing I though was that my BlackBerry might be low on memory, so I closed Maps.me, and tried again – no luck.  Then, I rebooted the phone, and took a few pictures when we were around the castle, but after that I couldn’t view my photos any more, none of the apps could start. Then I turned it off on my way back to hotel.

I tried many tests and combinations to revive the BlackBerry, because nowadays the smart phones mean everything to a person. I rarely take camera with me during most trips.  Everything is stored in the phone memory or in the cloud that need a smart phone to retrieve.  All my pictures since 22 months ago were on this phone, so were the videos, some documents, phone logs, contacts, even the key code to my hotel room.

So far these are the symptoms I found my phone had that night, and the next day, until I am back home.

  • The phone could reboot and power down using the button on the top, keyboard and touch screen were reactive.
  • After a fresh boot up, I could start a couple of apps, in a short period of time. Apps would stop working after 5-10 minutes.
  • I could see the hub, all email/message summaries, but couldn’t open any of them.
  • After 5-10 minutes, opening any apps would show a flash on the screen then app closed.
  • After reboot, some apps opened last time would show half tone on the home screen.
  • I could delete picture, remove apps when it’s reactive, but after reboot, they were still there.

I also did some research online, using my iPad, but I couldn’t do any soft reset, because I was scared that after reset it would start using roaming data. So basically, I didn’t have camera, GPS or map in the next two days in UK.

When back home, the first thing I wanted to do was to make a backup copy of all contents on the phone, so I connected it to my desktop, which has BlackBerry Link installed since I got this phone. The message I got on my PC was that the phone connected was not reachable, do you want to refresh the phone – aka factory reset?  I realized that I wouldn’t be able to get those new pictures back, but I need to revive the phone at least.  So I clicked Yes.

After a hour or so, the BlackBerry Classic was whipped clean, and everything looks fine, then I can start to re-customize my phone.

Lesson learned:

  1. Desktop sync software is helpful, it also makes backup of all pictures, videos etc when the phone was working.
  2. I should have made a backup using the Link software, I might be able to get more customized items back.

I saw a bunch of complaints online as well for the same problem, not sure if they have a solution yet since I checked last week. It may relate to the firmware update in February, but I believe the soft reset should resolve most of the issues.

 

Cloud, Microsoft

PowerShell for Azure Active Directory (AAD) and Office 365

Well, this is just some notes on how to prepare PowerShell to manage Azure Active Directory and Office 365. But similar to connecting to AD in Azure, you also need to go through these steps to connect PowerShell to an Azure subscription.

This is quite interesting actually when I put them together. As AAD still has two active environment versions, same as Azure console – Classic and RM – they belong to different logins, a little confusing to admins. Also PowerShell modules need to be installed and updated to enable different cmdlets set in order to manage different products – cloud, non-cloud, 3rd-parties like AWS, etc. So when something is not working, maybe you are in a wrong dimension or Microsoft wants you to update the binary you are using.

Azure Subscription AAD Module v1 (MSOnline)
– General Availability version
– Public Preview version
AzureAD v2
– General Availability version
– Public Preview version
Pre-requisites:
– Install lastest Azure Tools using Web Platform Installer
> Add-AzureAccount
> Get-AzurePublishSettingsFile
image
> Import-AzurePublishSettingsFile
> Select-AzureSubscription
Pre-requisites:
MS Online Services Sign-in Assistant
Windows AAD Module for Windows PowerShell
Pre-requisites:
PowerShellGet
image
Step 1: Install Azure PowerShell
Install-Module AzureRM
Download and install Azure AD Connect (.msi)
Update AAD Module > 1.0.8070.2
Download from PS Gallery
Install AAD Module
Update AAD Module
Step 2: Connect to an Azure account
Login-AzureRmAccount
Connect to Azure AD

$Msolcred = Get-credential
Connect-MsolService -Credential $MsolCred

Connect to AAD

$AzureAdCred = Get-Credential
Connect-AzureAD -Credential $AzureAdCred

Step 3: Run Azure PowerShell cmdlets

v1 cmdlets are here v2 are here
More information: PowerShell Gallery Office 365 PowerShell

One more thing: remember to check the update time of anything posted online, those older than 3 months might be useless.

Microsoft

How to enable telnet client using PowerShell

Telnet is a quite useful connecitivity verification tool, especially during the system set up or troubleshooting. For some reason, mostly security concern, this feature is disabled since Windows Server 2008. But when you want a quick check, it’s really a pain to open the Server Manager and click, click, click… Fortunately we have PowerShell. So here is the one-line command to turn it on:

Add-WindowsFeature telnet-client

And another command to turn it off, before you log-off:

Remove-WindowsFeature telnet-client

Very cool!

Microsoft

Windows 2012 R2 Preview Web Application Proxy – Exchange 2013 Publishing Tests

Need a close look, either continue to use MS technology, or switch to others after Forefront phase out.

Kloud Blog

==================
Updated: 10 September 2013

==================
Updated: 15 July 2013

  • I have heard from a member of the Web Application Proxy product group who said there is a bug in the Preview version that prevents Outlook Anywhere from working. They say it will be fixed in the RTM version
  • Lync 2013 and  Office Web Apps 2013 have been tested and work with some configuration changes. See http://blog.kloud.com.au/2013/07/15/publish-lync-2013-with-2012-r2-preview-web-application-proxy/ 
  • ActiveSync does not support SNI so a default binding needs to be set on the Web Application Proxy as per the post above to make it work
  • Clarification about ADFS being a hard requirement for the Web Application Proxy, even if only doing pass-through
  • Clarification about modifying published applications
  • Clarification about case sensitivity

==================

View original post 2,382 more words