Quantcast
Channel: Glen's Exchange and Office 365 Dev Blog
Viewing all 240 articles
Browse latest View live

How to Approve and Reject Moderation Emails in Exchange Online with the Microsoft Graph API and Powershell

0
0

A while ago I published this blog post about doing this using EWS and a few people have recently asked if it is also possible to do this with the Graph API(which it is) so I've decided to include this one in my Graph Basics series.

Moderation

Moderation is an Exchange feature that was introduced in Exchange 2010 that allows the Human control of mail flow to a distribution group or mailbox see https://docs.microsoft.com/en-us/exchange/security-and-compliance/mail-flow-rules/manage-message-approval for more detail.

Moderation Approve/Reject Messages

When a Message is requiring moderation an email is sent to one (or more) moderators requesting approval. In the Graph you can get these moderation messages by filtering on the MessageClass property. Because this isn't a first-class property like it was in EWS you need to use the singleValueExtendedProperties representation of the property. eg in the Graph a Request like this

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep:ep/id eq 'String 0x001a' and ep/value eq 'IPM.Note.Microsoft.Approval.Request')

To Approve or Reject a Message that requires moderation you need to do a few different things to create the approval/rejection Message

1. ItemClass- Set the ItemClass on the Message your sending in the Graph use the singleValueExtendedProperties like

Approval -

{
id = "String 0x001A"
value = "IPM.Note.Microsoft.Approval.Reply.Aprove"
}
Rejection -
{
id = "String 0x001A"
value = "IPM.Note.Microsoft.Approval.Reply.Reject"
}
2. Subject - Use the Normalized subject from the approval Request email and then prepend Accept or Reject 

3. RecipientTo - Needs to be set to the Microsoft Exchange Approval Assistant Address (Sender of the Moderation Message)

4. VerbResponse - This tells Exchange if you want to Approve or Reject the Message
{
id = "String {00062008-0000-0000-C000-000000000046} Id 0x8524"
value = "Approve"
}
5. PidTagReportTag Property - This is a critical property you get from the Approval Request which is used to correlate the Approval Response.
{
id = "Binary 0x31"
value = $ApprovalMail.singleValueExtendedProperties[0].value
}
Once you have all that information you can put a message together in JSON and send that using the SendMail endpoint in the Microsoft Graph


Invoke-ApproveModerationRequest will approve the last moderation request in a Mailbox 

Invoke-RejectModerationRequest will reject the last moderation request in a Mailbox


The MailboxConcurrency limit and using Batching in the Microsoft Graph API

0
0

If your getting an error such as Application is over its MailboxConcurrency limit while using the Microsoft Graph API this post may help you understand why.

Background  

The Mailbox  concurrency limit when your using the Graph API is 4 as per https://docs.microsoft.com/en-us/graph/throttling#outlook-service-limits . This is evaluated for each app ID and mailbox combination so this means you can have different apps running under the same credentials and the poor behavior of one won't cause the other to be throttled. If you compared that to EWS you could have up to 27 concurrent connections but they are shared across all apps on a first come first served basis.

Batching

Batching in the Graph API is a way of combining multiple requests into a single HTTP request. Batching in the Exchange Mail API's EWS and MAPI has been around for a long time and its common, for email Apps to process large numbers of smaller items for a variety of reasons.  Batching in the Graph is limited to a maximum of 20 items per batch.

Problems

When you make a batch request like the following against the Microsoft Graph to Get the child folders or particular root folders in a Mailbox (but it can be basically any other mailbox request)


If you have any more then 4 requests in the batch you will get a concurrency error for each request greater then 4. This goes back to the Mailbox concurrency limit being 4 by default, even though this is one request each of the requests in the batch gets executed asynchronous by default and this is what causes the limit to be exceeded. So the effective default batch limit for Mailboxes is 4.

If you wanted to have all the requests in your 20 item batch fulfilled in the one request you could use the dependsOn header eg


This makes the request fully sequential meaning that only one connection is ever opened to the Exchange Mailbox.

The problem with this is in a lot of real world scenarios its much slower eg enumerating all the folders in my mailbox using depends on with 20 Item batches took 14 seconds with 4 item batches async took 8. (note neither of these is good result vs a Deep Traversal in EWS  but there is currently no alternative in the graph). The batches weren't optimized because of the random hierarchy in my Mailbox but it would make sense that 4 cloud threads is going to win over 1 sequential one even with the greater number of client requests. 

So what should you do
  1. test,test,test to see what works best for you
  2. Make sure you always create a separate App registration for your apps (never reuse)
  3. Think about your context, if there is a chance your going to have multiple instances of your app running at the same time using the same user think about your batching strategy .
  4. Make sure you process the throttling responses, retrying your op at least once shouldn't be a big deal
My 2 cents

The way this API behaves in this scenario fails the pub test for me at the moment. If your using batching in EWS code then this isn't really the equivalent. eg someone spinning up 4 threads in EWS or MAPI to do the equivalent of what batching is doing in the Graph wouldn't be considered optimal (while they have tried to mitigate the need to use batching in the first place vs EWS and MAPI). The other side of the coin is its something you can exploit to gain some performance vs single op graph code.

 

Creating a Mailbox Search Folder based on a Message Category using the Microsoft Graph and Powershell

0
0

Searching on the Categories property of an Email can pose a challenge because this property is a Multi-valued String property (which aren't that common in email) eg in a Message the property may look like the following

image-20200917131803998

So this needs to be queried in a different way then a normal String or single valued property in an Email would, where you could use a number of filter options (eg equal, contains,startswith). In EWS it was only possible to query this property using AQS because of the way SearchFilters translated to the underlying ROP based restrictions used by the Exchange Mailbox Store. In the Microsoft Graph the Linq format in Filters does translate more favourably so can be used eg the following simple query can find Messages in a folder based on a specific category

https://graph.microsoft.com/v1.0/me/messages?filter=Categories/any(a:a+eq+'Green+Category')

you can create a SearchFolder based on this query which would search all folder in a Mailbox which would produce a SearchFolder Creation request like

{
"@odata.type": "microsoft.graph.mailSearchFolder",
"displayName": "Green Email",
"includeNestedFolders": true,
"sourceFolderIds": ["MsgFolderRoot"],
"filterQuery": "Categories/any(a:a+eq+'Green+Category')"
}

Based on a previous post on creating Search Folders I have this script https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/SearchFolder.ps1 which you can use to Create a Category Search folder as well as other SearchFolder CRUD operations eg

Invoke-CreateCategorySearchFolder -MailboxName user@domain.onmicrosoft.com -SearchFolderName CategoryTest -CategoryName "Category 1"

How to access and restore deleted Items (Recoverable Items) in the Exchange Online Mailbox dumpster with the Microsoft Graph API and PowerShell

0
0
As the information on how to do this would cover multiple posts, I've bound this into a series of mini post docs in my GitHub Repo to try and make this subject a little easier to understand and hopefully navigate for most people.  


The topics covered are




and the sample script is located 

Reporting on the Favorites Shortcut items in Outlook, OWA and Outlook Mobile using PowerShell and EWS

0
0

One of the email UI features that I find the most useful in Outlook on the Web and Outlook mobile is the People favorites feature which saves having to do a search for historical email from particular high use contacts. Favorites is a feature that has evolved especially in Outlook on the web and Outlook mobile eg People/Persona favorites and category favorites. The way this is implemented in the Mailbox is interesting eg   People/Persona favorites get their own search folder under the favoritePersonas Folder in the Non_IPM_Subtree in a Mailbox eg


As well as a configuration object under the 

\ApplicationDataRoot\32d4b5e5-7d33-4e7f-b073-f8cffbbb47a1\outlookfavorites eg


The configuration object is of interest as this tells as a lot about what type of favorites are being created and used in a Mailbox. It also can serve in a custom app if you want to reproduce the same type of favorites folder tree (you will need to use EWS for this as the Graph API is unfortunately hamstrung for this type of application). On the favourites object is a Extended property call RawJson that contains all the information that is of interest eg.



For this post I've created an EWS script that first finds the above folder and then gets all the Shortcut Items in that folder and retrieves the rawJson extended property on each of the ShortCut Items. It then does a little bit more data abstraction from the Json to make it into a report-able object. So for instance we can do a

Quick view of what Shortcuts are being used (which clients,Last-Visted(for groups)and type) eg


In the above Legacy = Outlook Desktop and Outlook Services is the Outlook Mobile client on Android

Just the Personal favorites


The script for this post is available on GitHub https://github.com/gscales/Powershell-Scripts/blob/master/Favourites.ps1


Looking at raw Mailbox analytics data using EWS and a ChangeDiscovery script

0
0

Mailbox analytics is something Microsoft have been working on for a number of years and its seems to be something that has received a little more effort in these pandemic times. If you have ever looked at the Non_IPM_Subtree of your mailbox you will see a lot of data being stored in their from various apps and substrate processes. A while back i wrote a ChangeDiscovery script to allow me to dump out quick what changes where happening in a Mailbox in a short time frame (eg i wanted to see what happened to all the items in a Mailbox when i performed a specific task). If you run this script with a slightly longer time-frame (eg looking over a day) it picks up all the Items that are being written and created for the Mailbox insights processes and other substrate processes.

Most of these emails get written under the 


Usually if I then wanted to look at these type of items I would use OutlookSpy or MFCMapi to browse the raw MAPI properties on items to see if they where of interest. Given the number of these items now and the performance on MAPI in Online mode doing this is extremely tedious so I modified my change discovery script further to dump the RawJson and Data properties that most of the insight items use so you can then look at the whole bunch in Excel. eg the following two properties


 So now i can dump basically everything that is happening in a Mailbox to a CSV in the last day and you can see some of this analytics data is pretty interesting  

eg


and if you looked at the data from the ActivitiesDaily


This is just a quick snippet of data on one item but things like the hourly breakdown of the number of the Messages being read per hour by a user is something that is really interesting and not something you can determine just using the raw data in a Mailbox.

While this raw data may surface elsewhere in a number of different guises and formats that are a lot of specific things you could use it for (outside of the narrative Microsoft are using for this).

The Modified change discovery script is available on Github https://github.com/gscales/Powershell-Scripts/blob/master/ChangeDiscovery.ps1

To run the script and look back 12 hours and dump the RawJson properties use something like

Invoke-MailboxChangeDiscovery -MailboxName gscales@mailbox.com -disableImpersonation -getJsonMetaData -secondstolookback 43200

for 24 hours use

Invoke-MailboxChangeDiscovery -MailboxName gscales@mailbox.com -disableImpersonation -getJsonMetaData -secondstolookback 86400




Finding Emails in a Mail Folder older then a specific date using the Microsoft Graph and Powershell

0
0

 


If you are doing any archiving, clean-up or just searching for Messages that are older then a specific date you will need to make use of a filter on the receivedDateTime.

The receivedDateTime property

This property represents when the Message was received expressed and stored in UTC time, this means when you query it you should also make sure your query value is in UTC.

So for instance if I where looking for all the Email in the JunkEmail folder older then 30 days I could use

/v1.0/users('gscales@datarumble.com')/MailFolders('JunkEmail')/messages?$Top=10&$filter=receivedDateTime lt 2020-10-21T00:00:00Z

If the mailboxes are in a TimeZone other then UTC then first convert the actual date to the local date you want to include to UTC eg in Powershell something like

(Get-Date).adddays(-30).Date.ToUniversalTime().ToString("o")

This means if my timezone is +11 UTC my actual query time would look like

2020-10-20T13:00:00.0000000Z based on an Actual day value of 2020-10-21T00:00:00.0000000+11:00

Precision

The last thing to consider around receivedDateTime is precision, Exchange Server stores the Item with a precision down to the milliseconds in EWS you could adjust the precision of your queries down to the millisecond however the Graph API doesn't allow you to do this. So if your querying items in the Graph and using the GT or LT operators you may see them work more as GE and LE due to the loss in precision. While this sounds like a little thing it can be quite painful depending on what your trying to do.

Orderby

By default when you use a filter you will get the results back in ascending order so if you want to see the latest messages first make sure you use

OrderBy=receivedDateTime desc

if you add more fields into the Filter clause they will also need to be added to the OrderBy

I've created a Powershell Script to go along with the doc call ItemAgeModule.ps1 available from https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/ItemAgeModule.ps1

Some Examples are get the last 10 email in the Inbox older then 30 days would look like

Get-EmailOlderThan -MailboxName gscales@domain.com -FolderName Inbox -OlderThanDays 30 -MessageCount 10

To get all the Email in the Junk-Email Folder older the 30 days use the following and it will page them back 100 items at a time

Get-EmailOlderThan -MailboxName gscales@domain.com -FolderName Inbox -OlderThanDays 30

To get all Email older then 30 days with a specific subject then you can use (note the OrderByExtraFields is needed which should have a comma seperated list of extra fields used in the filter)

Get-EmailOlderThan -MailboxName gscales@domain.com -FolderName Inbox -OlderThanDays 30 -filter "Subject eq 'test'" -OrderByExtraFields subject

Moving the Emails you find

As mentioned one of the main reasons you may have for finding messages of a specific age is to Archive, Move or Delete them. For this you need to use the move operation on the Item endpoint eg in graph this is basically a post like

https://graph.microsoft.com/v1.0/me/messages/AAK.....=/move

with the destination folder in the body of the POST (either a folderId or WellknownFolderName) eg

{  "destinationId": "archive" }

Would move messages to the mailbox's archive folder (not my in-place archive)

So if I wanted to archive all the Messages in my Inbox from a particular sender that was older then 2 years I could use

Move-EmailOlderThan -MailboxName gscales@domain.com -FolderName Inbox -OlderThanDays 30 -filter "from/emailAddress/address eq 'user@domain.com'" -OrderByExtraFields "from/emailAddress/address" -Destination archive

This code makes use of batching and moves items in lots of 4 as not to go over the concurrent user quota

Or if I wanted to soft delete items I could move them to the RecoverableItemsDeletions folder

eg

Move-EmailOlderThan -MailboxName gscales@domain.com -FolderName Inbox -OlderThanDays 30 -filter "from/emailAddress/address eq 'info@twitter.com'" -OrderByExtraFields "from/emailAddress/address" -Destination RecoverableItemsDeletions -verbose

the script for this post can be found https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/ItemAgeModule.ps1

Back to 101 Graph PowerShell Binder Mini-Site

Getting Teams Chat History Messages (Compliance Records) From a Mailbox using EWS and/or in Outlook Add-in

0
0

 Microsoft made a recent change to where the compliance messages get stored in a Mailbox for private Teams Chats from the \Conversation History\Teams Chat Folder to a non_ipm_subtree folder called TeamsMessagesData (see https://office365itpros.com/2020/10/14/microsoft-changes-location-teams-compliance-records/ for some good commentary on this change). In terms of programmatic access to these messages this change affects the ability of the Graph (and Outlook REST) Endpoints to access them. While you could never access the Folder directly with the Graph API (because of the FolderClass) the messages themselves would appear in the \Messages endpoint (this is because they where captured by the underlying AllItems Search folder that backs this endpoint). Now the messages are in the Non_IPM_Subtree you loose the ability of that Search Folder to access those messages. This first up broke my OWA add-in from https://gsexdev.blogspot.com/2019/03/microsoft-teams-private-chat-history.html that could be used to show the Chat History for a user using the compliance messages in a Mailbox  eg



While the Graph and Outlook REST Endpoints can't be used to access these compliance records anymore (unless you create your own search folder to expose them). You can use EWS to still access them, to fix my Outlook Add-in this meant switching the code from using the Outlook Rest Endpoint to EWS to accessing these message. I also wrote a script to show how these message could be accessed via powershell

There are two ways of getting the TeamsMessagesData Folder in EWS the first is you can use an Extended property that exists on the Inbox folder that will contain the hex entryId to the TeamsMessagesData Folder you then need to convert that to an EWS id and then bind to the Folder eg (In the script I've done this)


The other way is just find the folder using its name in the Root of the Non_IPM_Subtree. (In the Plugin I do this)


With the script it means is I can do something like this to view the Chat messages in a Mailbox (last 30 days) eg


As well as changing the Add-in from using the Outlook Rest endpoint to EWS I've also included a look back dropdown so if you want to see more then the default 60 days of messages you now can eg


(what you will see will depend on the Retention policy for your compliance records ).

The script is available from https://github.com/gscales/Powershell-Scripts/blob/master/GetTeamsChatMessages.ps1

With the Add-in is I've hosted the files on my GitHub pages so its easy to test (if you like it clone it and host it somewhere else). But all you need to do is add it as a custom addin (if your allowed to) using theURL

  https://gscales.github.io/TeamsChatHistory/TeamsChatHistory.xml



The GitHub repository for the Addin can be found here https://github.com/gscales/TeamsChatHistoryOWAAddIn




Using Shared Mailboxes in the Microsoft Graph API from PowerShell

Auditing Inbox rules (and looking for hidden rules) with EWS in OnPrem Exchange

0
0

 After the events of the last weeks around the latest zero day vulnerabilities in Exchange  and once you've finished cleaning up any back doors that may have been left on servers its a good idea to review some other less known but established ways bad actors may hide persistent access within Mailboxes. One of these are Inbox Rules (but Mail Flow rules could also be used) and a more advanced method is the hidden Inbox rule exploit that was first talked about https://blog.compass-security.com/2018/09/hidden-inbox-rules-in-microsoft-exchange/ and I covered it in https://gsexdev.blogspot.com/2019/05/audting-inbox-rules-with-ews-and-graph.html and somebody else https://mgreen27.github.io/posts/2019/06/09/O365HiddenRules.html there are a number of tools and techniques around detecting these types of rule but are all focused more toward Office365 as that was where at the time this exploit was being mostly employed. In my post at the time I modified the Microsoft script https://github.com/gscales/O365-InvestigationTooling/blob/master/Get-AllTenantRulesAndForms.ps1 so it would include the PidTagRuleMsgProvider property in its final report so you could easy spot any null, empty or custom values that would effectively hide rules from being enumerated via other more conventional methods. Because this tool was designed for o365 it can't easily be run against onPrem Exchange so I've put together a modified version that can be and posted it up here.  https://github.com/gscales/Powershell-Scripts/blob/master/EWShiddenRuleEnum.ps1  I've retained the funky Encoded compressed version of the EWS Managed API dll that the original used and extended some of the properties to include things like the display-name of the user and the last modified date time of the rule object. To use this script you need to have EWS Impersonation enabled for the user you either pass into the script or the user the script is running under. https://docs.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-configure-impersonation

Mostly the changes to the script involved (other then cleanup) where around the remote power shell differences (this script will try to open a remote ps connection if one isn't already established) and passing in the hostname of the EWS endpoint.

To use this script first import the script module

Import-module ./EWShiddenRuleEnum.ps1

Then 

Invoke-EnumerateAllInboxRules

parameters that can/must be used

EWSHostName this is the EWS endpoint hostname the script doesn't use autodiscover so if your not running directly on the destination server you must pass this in

ExchangePSHostName this is the servername that the Remote powershell session will be open against if your not running directly on the destination server you must pass this in

Credentials are the credentials that will be used in EWS to connect and impersonate mailboxes, the remote powershell credentials used are the running users. (credentials can be entered in either UPN format or downlevel format domain\username and this can be environment dependent as to what works)

Filter this allows you to filter the Get-Mailbox query with any valid filter you would normally use in the PowerShell eg Invoke-EnumerateAllInboxRules -filter '{servername -eq hostname}'

The script will produce a CSV report of rules and forms in the current directory, you can open this in Excel and apply filters to make it easy to read and interpret. 

If you don't go down the path of rebuilding Exchange servers that where exploited make sure you also look at any Transport Agents you have installed and verify the assemblies, its a a lesser known method of persistent access but it has been used before https://www.bankinfosecurity.com/researchers-spies-exploit-microsoft-exchange-backdoor-a-12459 to some success.   




Using the Tag for external email messages received feature in the Microsoft Graph and Exchange Web Services

0
0
The "Tag for external email messages received" feature was introduced into Office365 recently to help people better to identify mail that comes from external sender vs internal sender see this for more info on this feature .

What happens when you enable this feature is that for messages with External sender a new Mapi property is set on those messages eg


For messages from internal senders the property doesn't appear to get set and if the feature isn't enabled in your tenant then you won't see this property either. You can negate the boolean value of the property which will turn off the external tag.

Using it in the Microsoft Graph API


If you want to use it in the Microsoft Graph API when you retrieve messages you can include this as a SingleValueExtendedProperties eg for Retrieving and filtering message you can use


I've included this in my Get LastEmail 101 graph sample if you want to try this in a real mailbox eg you can do

 Get-LastEmail -InternalSender -MailboxName gscales@mailbox.com 

Will get the Last mail from an Internal Sender

 Get-LastEmail -ExternalSender -MailboxName gscales@mailbox.com 

Will get the Last mail from an External Sender

and you can use the -Focused or -Other switch to get the above email from the Focused or Other Inbox

Using it in Exchange Web Services


In EWS you can use a SearchFilter eg

This property is set-able so if you have an application where email is marked as external and you can't or don't want to whitelist the domain (so they no longer are marked external) you could modify this property value (on each applicable message) so they will no-longer be tagged as External for the user.

Microsoft Teams Call History Outlook/OWA Addin

0
0

One of the Microsoft Teams features that I like the least is call history, for a number of reasons firstly its quite limited only 30 days worth of data and only includes direct calls and not meetings (which are often just scheduled calls). So when it comes to actually answering a question (like the one I found myself asking last week) when did I last have a call or meeting with person X it can't be used to answer this question.  The other thing is having that information available in the context of Outlook/OWA is a lot more useful for me (anyway) then digging through anther client to find its doesn't answer the question anyway. However while the call history in Teams only goes back 30 days the Teams CDR (Call Data Records) which I covered in https://dev.to/gscales/accessing-microsoft-teams-summary-records-cdr-s-for-calls-and-meetings-using-exchange-web-services-3581 do go back as far as your mailbox folder retention period. So with a little work I can use Exchange Web Services to create an Outlook/OWA addin that can be used to show the Call history for a person inline in OWA when you active it on a message you received from that user Eg


How this works is that the CDR item that is stored in the Mailbox stores the participants of the Call or Meeting in the recipients collection of the CDR mailbox item. The means you can make a KQL query on the participants to find all the CDR items that involve a particular user based on the sender address of the Email that the OWA Addin is activated on. eg the EWS Query ends up looking like


The extended properties that are used for the Start and End are pidTagStart and pidTagEnd which get set on the CDR mailbox item and then the SenderEmailSMTP address is used to work out who is the Organizer or the call initiator. To work out the type of Call the ItemClass is used so the last part of the ItemClass will tell you what the CRD is for eg call or meeting (If your wondering if you could do the same with the Microsoft Graph the answer is no because the Items in question are not a subclass of IPM.Note so aren't accessible when using the Microsoft Graph)

I've posted the code for the Addin in the following GitHub repository https://github.com/gscales/Microsoft-Teams-Call-History-Outlook-OWA-Addin

Want to give it a try yourself ?

I've hosted the files on my GitHub pages so its easy to test (if you like it clone it and host it somewhere else). But all you need to do is add it as a custom addin (if you allowed to) using the

URL- https://gscales.github.io/TeamsCallHistory/TeamsCallHistory.xml




Using Out of Office / automaticRepliesSetting with the Microsoft Graph with Service Principal Authentication

0
0

Out of Office (or automaticRepliesSetting) can be used for a vast number of different applications. For example in this Teams In/Out board 


With the Microsoft Graph API there are two ways that can be used to get the automaticRepliesSetting either via the Mailbox setting Endpoint eg

https://docs.microsoft.com/en-us/graph/api/resources/automaticrepliessetting?view=graph-rest-1.0



Or you can use MailTips which was the method i used in the Teams Apps eg


eg


When it comes to setting the OOF you must use the Mailboxsettings endpoint

What is better ? for getting the OOF settings on a large number of users getmailtips because you can request up to 100 users in one request while if your batching Mailboxsetting you can only have a max of 20 user in a single batch.

Permission and Authentication 

One consideration for the Mailboxsettings endpoint is there is no ability to use Delegate permissions to access the Mailbox settings of a user other then that of the credentials (unlike EWS where you could use delegation). So for most use cases (eg daemon apps or other automation back-ends) you will need to use Service Principal authentication and Application permissions. This can open up some security concerns if the application in question is only required to work on a subset of Mailbox make sure that you employ scoping 

Using Service Principal Authentication in Power Shell

I don't yet have a sample for doing this in my Graph 101 series but its actually less complicated then doing an authentication code type auth because all your code needs to do is create a JWT token from a locally stored SSL certificate (and sign it). Here is some sample code for doing this


So to use this function you first need to acquire the local certificate which could be either stored in the Local Certificates Store or in a PFX file with a password. eg

$certificate = Get-ChildItem -Path Cert:LocalMachine\MY\FCE7328421DDDB4CC8EA59C51B6F156765272CFA

or from a PFX file

$certificate = Get-PfxCertificate -FilePath C:\temp\certo.pfx

Once you have the certificate you can then request the token like

Get-AccessTokenForGraphFromCertificate -Certificate $Certificate -ClientId e6fd6f09-9c63-4b30-877a-3520ee1e1e9a -TenantDomain domain.com

Samples for Getting the AutoReplySettings using Service Principal Authentication

For this post I've put together a sample of Getting the AutoReply Setting for One user and batch of users via the Mailbox Setting Endpoint using certificate Authentication. And also a sample or using MailTips to do the same.




Using Batching to improve the speed of Contact creation in the Microsoft Graph

0
0

There's been a few contact creation scripts popup recently for the Graph API like this as well as a few questions on the forums around this topic lately. None of these examples and questions are taking advantage of using batching in the Microsoft Graph which will give you a significant uplift in performance vs the single request method when creating larger numbers of items and also help you a little around throttling.

I've added a new post to my Graph 101 binder on GitHub that includes an example of doing a CSV Contact import using batching and Service Principal Authentication https://github.com/gscales/Graph-Powershell-101-Binder/blob/master/Contacts/Batch%20Importing%20Contacts.md 

If your interested in a EWS version that can use larger batches (eg 60-100 contacts per request) I've also include an example on GitHub for this https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/BatchContactCreationEWS.ps1

Migrating Exchange Web Services (EWS) Directory and Recipient resolution code to the Microsoft Graph

0
0

 One of the more complex things to migrate in EWS when migrating to the Graph API is any directory access code that uses one of the following EWS operations

  • FindPeople
  • ResolveName
  • ExpandGroup (ExpandDL)
or if your using OnPrem you maybe using System.DirectoryServices to do direct LDAP requests of Active Directory.

With the Microsoft Graph API these Directory based mail operations don't exist, because you have full access to the underlying AzureAD, so in theory everything should be achievable without these type of operations. For the most part this is correct where is starts to get a little grayer is around this like Address Lists and Exchange recipient types mostly because the Graph doesn't expose the following underlying Active Directory properties

  • msExchRecipientDisplayType
  • msExchRecipientTypeDetails
  • msExchRemoteRecipientType
so this can be a limitation if your migrating from LDAP code and some FindPeople implementations if your searching based on AddressList.

Microsoft Graph Users Endpoint and the People API

The two main endpoints you will use when migrating your directory based code is the Users endpoint which allows you to enumerate and query the underlying users in Azure AD.  The People API allows you to query both the Users, Mailbox Contacts, Directory and other related contacts information (and a little bit more). The third endpoint you might use is Contacts if you just want to search the contacts of the local mailbox but I'm not going to cover this in this post also the Groups Endpoint should be used for group operations (expanddl)

Permission and uses 

For the People API there are two permissions that are usable but if you need to be able to search all users in your organization (regardless of their relevance to the underlying user) you need to make sure that you have the People.Read.All permission that requires Admin Consent. For the Users Endpoint depending on the level of detail you need user.readbasic.all is an option but you may find you need user.read.all.

Sample Scripts

For this post I've created a number of sample scripts for the user endpoint i have 


(users Delegate Authentication) 

https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/EnumerateUsersSpAuth.ps1

(Uses Service Principal Authentication)

https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/PeopleModule.ps1

Users API Advanced Queries

The Microsoft Graph API recently introduced an Advanced query engine that allows you to make more complex queries of Azure AD https://docs.microsoft.com/en-us/graph/aad-advanced-queries . To use this advanced query the Request string needs to modified and a additional header 

ConsistencyLevel: eventual

needs to be added to the Graph request, this is supported in the sample script for using the -AdvancedQuery switch

People API Search Scope

As well as the permission scope in the People API the scope it will search can be affected by an additional header called 

"X-PeopleQuery-QuerySources: Mailbox,Directory”

By default the People API will only search the Mailbox so it's important to use this header when you want to search the Directory or even limit your search to the Directory (and don't search the mailbox). In the sample script using the -DirectoryOnly switch will limit the search to the directory only. One thing to note if you don't have the People.Read.All permission and you use a Directory only search scope it will only return those contacts relevant to the user rather then rather then any user in the directory.

Enumerating or browsing users (Find People)

Address books are a pretty standard feature of any email client, in EWS you may have used the FindPeople operation to search or browse the users from an Addressbook. The good thing about findpeople is that it gave you the ability to use Exchange Address Lists. The Graph doesn't have that direct ability so if your trying to browse using Address lists using one of the Graph endpoints to try and mimic the GAL you need to try to reproduce that filter in your code (which you may not be able to do).  When you enumerate objects in the Graph API you page those objects in pages of  up to 999 objects at a time. An example of a straight page of all the users in the underlying directory is

Get-AzureUsersFromGraph -MailboxName gscales@datarumble.com

Using the People API you can search across all Exchange Address Lists (rather then browse within a particular address lists like FindPeople could). Because this is a search rather then an enumeration like the Users endpoint you can use a workaround such as search for "smtp:" as the search string to return all users (this was a workaround for EWS as well it was limited to 100 users in EWS I haven't tested if there is the same limit in the Graph and its not documented, the differences is the Microsoft Graph results are paged where resolveName wasn't, paging is also a little weird in the People API and the documentation is lacking at the moment and doesn't explain how it works (or it has bugs at the moment) (eg Directory queries look like there is a limitation to 150 where Mailbox queries aren't limited ) If anybody can give clarity around this let me know and I'll correct this.

Invoke-SearchPeople -MailboxName gscales@datarumble.com -SearchString 'smtp:' -DirectoryOnly

I've added a Graph 101 Binder page for enumerating users in the directory which has more example for the users interface https://github.com/gscales/Graph-Powershell-101-Binder/blob/master/Users/Enumerate%20and%20Search%20for%20users%20in%20Azure.md

ResolveName

In EWS the ResolveName operation has a number of different uses in the conventional sense it would be used to do ANR (ambiguous name resolution) when someone is sending email. In an Application it maybe used to do any of the following 

Get the contact information for a user that you only have the SMTP Address for

If the target users are located in AzureAD you could use the Users Endpoint in Azure to search for the user based on one of their proxyaddress so it could be a Primary or secondary address of the object. eg

Get-AzureUsersFromGraph -MailboxName gscales@doamin.com -filter "proxyAddresses/any(x:x eq 'smtp:info@domain.com')"

Which generates a Request to Graph like
https://graph.microsoft.com/v1.0/users?$top=999&$filter=proxyAddresses/any(x:x%20eq%20'smtp:info@users.com')

If the Contact your trying to locate is in the Mailbox or if its in either the Mailbox or the Directory then you can use the People API to search both scopes. Note this is a Search so you may need to filter the results more then you would in the Users example with is doing an exact match

Invoke-SearchPeople -MailboxName gscales@datarumble.com -SearchString 'smtp:info@domain.com' -DirectoryOnly

Find All users from a particular domain (eg all guest users from domain)

Get-AzureUsersFromGraph -MailboxName gscales@datarumble.com -filter "endsWith(mail,'@msgdevelop.com')" -AdvancedQuery

Which generates a Request to Graph like (notice this is an Advanced Query)
https://graph.microsoft.com/v1.0/users?$top=999&$filter=endsWith(mail,'@msgdevelop.com')&$Count=true

Search based on part of a DisplayName 

For the closest ResolveName behavior use the People API however the Users Endpoint in Graph can also be used here are some examples

Users Endpoint

 Get-AzureUsersFromGraph -MailboxName gscales@datarumble.com -filter "startswith(displayName,'glen')" -AdvancedQuery

People API

Invoke-SearchPeople -MailboxName gscales@datarumble.com -SearchString 'glen' -DirectoryOnly

Resolving a Legacy ExchangeDN or EX Address

This is something I've used a lot especially when your migrating email or in other instances where all you have is the LegacyDN or Exchange Native Address of the User. You can resolve these addresses using the People API and a DirectoryOnly query eg

Invoke-SearchPeople -MailboxName gscales@datarumble.com -SearchString '/o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=a7d0ec70c15e4132a8a2cfb840c75a66-gscales' -DirectoryOnly

Expand Group

I haven't actually included this in any of my samples as there is a Groups Endpoint in the Microsoft Graph that you should use for this which is documented well at  https://docs.microsoft.com/en-us/graph/api/resources/groups-overview?view=graph-rest-1.0

Sending a MimeMessage via the Microsoft Graph using the Graph SDK, MimeKit and MSAL

0
0

One of the new features added to the Microsoft Graph recently was the ability to create and send Mime Messages (you have been able to get Message as Mime for a while). This is useful in a number of different scenarios especially when trying to create a Message with inline Images which has historically been hard to do with both the Graph and EWS (if you don't use MIME). It also opens up using SMIME for encryption and a more easy migration path for sending using SMTP in some apps.

MimeKit is a great open source library for parsing and creating MIME messages so it offers a really easy solution for tackling this issue. The current documentation on Send message via MIME lacks any real sample so I've put together a quick console app that use MSAL, MIME kit and the Graph SDK to send a Message via MIME. As the current Graph SDK also doesn't support sending via MIME either there is a workaround for this in the future my guess is this will be supported.



Sending a Message with a Large attachment using the Microsoft Graph and Powershell

0
0

Sending a message with a large attachment used to be the least worst option when it came to shipping data between people. In the Modern workplace where document and file sharing options are a lot better sending email with a large attachment can tends now to be the worst option especially when it comes to version control, expiration etc. However if you do find yourself with the need to send an email with large attachments (larger then 4MB) and you want to make use of the Microsft Graph API then it requires you use a different method from just sending the message in one post eg https://docs.microsoft.com/en-us/graph/outlook-large-attachments?tabs=http

This does make sending a Message a lot more complex eg if you have a Message less the 4MB you can just make one REST Post to send that message. If you have a large attachment you then have a multi step process eg

  1. Create a Draft message where you set all the other message property eg Subject,To,Body
  2. Take the MessageId(graph item id) returned in Step 1 and Create an Upload Session
  3. Read the attachment and upload the attachment in chunks until you have uploaded the entire file
  4. Send the Draft Message
Eg this is what it looks like in term of REST requests


I've put together a Powershell sample and posted it on GitHub  https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/SendMessageWithLargeAttachment.ps1 that performs the above to run this use something like

Invoke-SendMessageWithLargeAttachment -MailboxName gscales@....com -filePath "C:\temp\rates.pdf" -verbose -uploadChunkSize 400000 -Subject "Test Message 1234" -body "Rgds Glen" -To glenscales@yahoo.com

The chunk size you use is up to you a larger chunk size will generally mean a faster upload but if the Network is more unreliable a smaller chunk is better




Using MSAL in the EWS Managed API and doing auto token expiration and renewal in Delegate and Client Credential Azure oAuth Flows

Migrating your Mailbox searches in EWS to the Graph API Part 2 KQL and new search endpoints

0
0
This is part 2 of my blog post on migrating EWS Search to the Graph API, in this part I'm going to be looking at using KQL Searches and using the new Microsoft Search API (currently in Beta). The big advantage these type of searches have over using SearchFilters is that these type of searches use the content indexes which can improve the performance of searches when folder item counts get high. They also allow you to query the contents of  Attachments which are indexed through ifilters on the server.

KQL queries on the Mailbox and Mailbox Folders

In EWS you have been able to use firstly AQS and now KQL in the FindItems operation from Exchange 2013 up. To migrate these searches to Microsoft Graph is pretty simple eg an EWS FindItem query to search for all messages with a pdf attachment

FindItemsResults fiItems = service.FindItems(QueryFolder,"Attachmentnames:.pdf", iv);

in the Graph you would use something like

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages
?$search="attachmentnames:.pdf"


the slightly disappointing thing with the Graph is that you can't use count along with a search which when your doing statistical type queries eg say I wanted to know how many email that where received in 2019 had a pdf attachment makes this very painful to do in the Graph where in EWS it can be done with one call (its a real snowball that one).

Searching the recipient fields like To and CC, in the forums you see some absolute clangers search filters that try to search the recipients and from fields of messages that can easily be done using the participants keyword which includes all the people fields in an email message. These fields are From, To, Cc. The one thing to be aware of is the following note on expansion in https://docs.microsoft.com/en-us/microsoft-365/compliance/keyword-queries-and-search-conditions?view=o365-worldwide . So if you don't want expansion to happen you need to ensure you use the wildcard character after the participant your searching for. A simple participants query looks like

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$search="participants:Fred"


Date range queries

One of the good things about KQL with dates is that you can use reserved keywords like today,yesterday,this week eg

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')
/messages?$search="received:yesterday"


to get all the received sent between two dates you can use either

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$search="received:2019-01-01...2019-02-01"


or

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$search="(received>=2019-01-01 AND received<=2019-02-01)"


If you want to search the whole of the Mailbox using the graph eg if you have use the AllItems Search Folder in EWS to do a Search that spans all the MailFolders in a Mailbox in the Graph you just need to use the /Messages endpoint eg

https://graph.microsoft.com/v1.0/me/messages?
$search="(received>=2019-01-01 AND received<=2019-02-01)"



New Search Methods

The traditional search methods in EWS give you the normal narrow refiner search outputs that most mail apps have been providing over the past 10-20 years. While these methods have improved over the years there hasn't been any real great leaps in functionality with Search. So the Microsoft Graph has been adding some newer endpoints that do allow a more modern approach to searching . The first is Microsoft Graph data connect which has been around for a while now and the Microsoft Search API which is still in Beta. As this article is about migrating EWS searches you probably wouldn't consider either of these for your traditional search migration as $filter and $search are going to meet those needs. However if you are looking at overhauling the search functionality in your application or you are building greenfield functionality then both of these new methods are worth consideration.



Graph Data connect is your go-to endpoint when you want to do any mass processing of Mailbox data. It solves that problem of having to crawl every item in a Mailbox when you want to do any data-mining type operations by basically providing an Azure dataset of this information for you. Data connect is great however it has a high entry level, first you need a Workplace analytics licence for every mailbox you wish to analyse and the costs can mount pretty quickly the larger the Mailbox count your dealing with. The other requirements is paying for the underlying Azure Storage etc that your dataset ends up consuming. I think it can be a bit of a shame that the licencing costs can lock a lot of  developers out of using this feature as it really does provide a great way or working with Mail item data. And it leaves some having to resort to doing their own crawling of Mailbox data to avoid these costs (eg that licencing cost is a pretty hard sell for any startup looking to use this) 


Microsoft Search API

https://docs.microsoft.com/en-us/graph/search-concept-overview

This is the newest way of searching mailbox data, while the underlying mechanism for doing mailbox searches is still KQL so its very similar to the $Search method described about,  this API does enhance the search results with some more "Search Intelligence" like relevance bringing AI into the picture . One of the other main benefits of this endpoint is when you want to broaden your search to other Office365 workflows or even include your own custom data searches. So this really is the endpoint that will provide you with a modern search experience/workflow. Which is getting more critical due to the sheer amount of data we have (eg the datageddon). Its still in beta and is a little restricted at the moment eg

  • It can't be used to search delegate Mailboxes so only the primary mailbox 
  • It only returns the pageCount for items not the Total number of Items found in a search (to be fair $search does this as well which is really annoying)
  • Searches are scoped across the entire mailbox 
  • Just Messages and Events are searchable at the moment






Important change to EWS access to Microsoft Teams data

0
0
Microsoft have annouced they will be restricting access to Microsoft Teams data via EWS from the 30th September see https://devblogs.microsoft.com/microsoft365dev/restricted-access-to-microsoft-teams-data-via-ews-starts-september-30-2022/ . If you are using EWS to access any teams data you will need to move to the Graph API's https://docs.microsoft.com/en-us/microsoftteams/export-teams-content one thing to note is "Microsoft Teams APIs in Microsoft Graph that access sensitive data are considered protected APIs" so there are some extra steps and time you need to access this data and be aware of the potential need to pay for that access eg "On July 5th 2022, metered consumption is active, and will be billed through an Azure subscription" ref https://practical365.com/teams-export-graph-apis/.
Viewing all 240 articles
Browse latest View live




Latest Images